VLA Gallery / All models / Humanoid whole-body controllers

ExBody2

UC San Diego·UC Berkeley·MIT · 2024-12

VAHumanoid whole-body controllersNot disclosedRL policy

Automatically filters training motions with a tracking-error threshold τ* = 0.150 to track expressive whole-body motions.

Paper

Backbone LLM: None
Vision encoder: —
Action head: Joint PD targets (RL policy)
Chunk: Not disclosed
Control rate: 50 Hz (low-level 500 Hz)
Architecture (original summary): MLP: teacher actor·value [512, 256, 128], student [1024, 1024, 512], proprioceptive history 10; motion targets + proprioception → joint PD targets, policy 50 Hz·low-level 500 Hz

Pretraining accelerators: Not disclosed (IsaacGym)
Pretraining duration: Not disclosed
Pretraining steps & batch: Not disclosed (env count Not disclosed)
GPU/TPU-hours: Not disclosed
Fine-tuning: N/A
Data type: Motion capture
Data scale: CMU motion capture 1,919 sequences (before filtering); size after filtering not disclosed
Main datasets: CMU motion capture
Tasks & evaluation: MPKPE·MPJPE tracking error compared against 4 baselines
Robot: Unitree G1(23 DoF)

Inference device & speed: Policy 50 Hz, low-level 500 Hz, communication latency 18–30 ms; onboard Jetson Orin NX
Inference memory: Not disclosed
Weights bf16 (calc.): Not disclosed
Where it runs: Onboard (Jetson Orin NX)

Reinforcement learning: Used (RL controller training) · RL teacher (privileged information, actor-critic) → student distillation
Base control: Humanoid legged (RL policy drives joints directly) · Tracks whole-body keypoint and root velocity goals, outputting 23-dim joint PD targets including legs (walking, crouching, dancing)

Key features: Automatically filters training motions with a tracking-error threshold τ* = 0.150 to track expressive whole-body motions. Runs onboard on Jetson Orin NX.
Notes: Selection error e(s) = 0.1·E_key + 0.9·E_dof (extraction). Not covered by the verification file.
Verification: Not verified