VLA Gallery / All models / Humanoid whole-body controllers

TWIST

TWIST (Teleoperated Whole-Body Imitation System)

Stanford University·Simon Fraser University · 2025-05

VAHumanoid whole-body controllersNot disclosedRL policy

Performs mocap-based whole-body teleoperation with a PPO privileged teacher and an RL+BC jointly optimized student.

Paper

TWIST: Teleoperated Whole-Body Imitation SystemarXiv 2505.02833

Overview

Category

Humanoid whole-body controllers

Type

VA

Released

2025-05

Organization

Stanford University·Simon Fraser University

Open weights

Yes

Parameters

Not disclosed

Architecture

Backbone LLM

None

Vision encoder

—

Action head

Joint position commands (RL policy)

Chunk

Not disclosed

Control rate

50 Hz (PD 1000 Hz)

Architecture (original summary)

MLP [512, 512, 256, 128] (SiLU, LayerNorm) + Conv1d motion encoder (latent 128), history 10; mocap reference → joint position commands 50 Hz, PD 1000 Hz; Transformer blocks in the repo are commented out

Training

Pretraining accelerators

RTX 4090 × 1 (README)

Pretraining duration

1~2 days (README)

Pretraining steps & batch

Teacher max_iterations 30_002, 4096 envs (repo)

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A

Data type

Motion capture

Data scale

AMASS · OMOMO 15,000+ clips (about 42 hours) + in-house mocap 150 clips (about 0.5 hours)

Main datasets

AMASS, OMOMO, in-house motion capture

Tasks & evaluation

Teleoperation of whole-body manipulation, leg-assisted manipulation, locomotion, and expressive motions

Robot

Unitree G1 (29 DoF); Booster T1 sim only

Inference

Inference device & speed

Joint position commands 50 Hz on RTX 4090, PD 1000 Hz; OptiTrack mocap 120 Hz; teleoperation latency about 0.9 s

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Local PC (RTX 4090)

Reinforcement learning & base

Reinforcement learning

Used (RL controller training) · PPO teacher → RL·BC jointly optimized student

Base control

Humanoid legged (RL policy drives joints directly) · Tracks mocap reference (joint positions·root velocity) → whole-body joint position commands at 50 Hz (side-stepping·walking backward·crouching through·kicking)

Notes

Key features

Performs mocap-based whole-body teleoperation with a PPO privileged teacher and an RL+BC jointly optimized student. Trains in 1~2 days on a single RTX 4090.

Notes

Training GPU and time are not in the paper; values from README (confirmed in verification). Full release 2025-09-29 (motion data, teacher and student training code, sim2real scripts, general motion tracker checkpoint). Data hours reflect only about 42 hours of public data; about 0.5 hours of in-house data separate.

Verification

Confirmed