VLA Gallery / All models / Humanoid whole-body controllers

BeyondMimic

UC Berkeley·Stanford University · 2025-08

VAHumanoid whole-body controllersDiffusion policy 19.95Mdiffusion

Adaptive sampling learns the hard segments of long motions, transferring LAFAN1 motions to hardware without parameter tuning.

Paper

Backbone LLM: None
Vision encoder: —
Action head: Diffusion planner (6-layer transformer, 19.95M) (diffusion)
Chunk: Not disclosed
Control rate: Not disclosed
Architecture (original summary): MLP tracker (actor·critic [512, 256, 128], ELU): proprioception + reference motion → joint targets / guided diffusion policy: Transformer decoder 6 layers·4 heads·512 dims

Pretraining accelerators: Not disclosed (Isaac Lab v2.1.0, repo)
Pretraining duration: Not disclosed
Pretraining steps & batch: max_iterations 30000 · 24 steps per env, 4096 envs (repo); sim dt 0.005, decimation 4
GPU/TPU-hours: Not disclosed
Fine-tuning: N/A
Data type: Motion capture / simulation·synthetic
Data scale: About 2.5 h of human motion (v4); 30 clips (15 min) for real-robot deployment; diffusion policy data are rollouts of domain-randomized expert policies (scale not disclosed)
Main datasets: Unitree retargeted LAFAN1, prior-work datasets, online animation data
Tasks & evaluation: Unitree G1 cartwheel·sprint·360° spin jump; diffusion policy for waypoint navigation·joystick teleoperation·obstacle avoidance (50 trials per joystick experiment)
Robot: Unitree G1

Inference device & speed: Tracker: robot CPU ONNX Runtime, under 1.0 ms per step / diffusion policy: offboard RTX 4060 Mobile TensorRT, 20 denoising steps about 20 ms
Inference memory: Not disclosed
Weights bf16 (calc.): Not disclosed
Where it runs: Offboard laptop + robot CPU

Reinforcement learning: Used (RL controller training) · MLP tracker trained with Isaac Lab actor-critic RL (4,096 envs). Diffusion planner uses supervised learning
Base control: Humanoid legged (RL policy drives joints directly) · Tracking policy follows reference motion and directly outputs whole-body joint PD targets (walking, running, cartwheels); diffusion policy generates locomotion via joystick velocity and waypoint cost guidance

Key features: Adaptive sampling learns the hard segments of long motions, transferring LAFAN1 motions to hardware without parameter tuning. The diffusion policy follows waypoint, joystick, and obstacle-avoidance goals via test-time guidance.
Notes: Parameters (B) is null since total parameters are not disclosed (only the 19.95M diffusion policy is disclosed; tracker not disclosed). Architecture is transformer per the diffusion planner; the tracker is an MLP. Per-motion convergence iterations (Cristiano Ronaldo 1.5k, dance 8k~10k)†. The earlier HTML version referenced 40 LAFAN1 motions (several minutes each). arXiv v4 2025-11-13. Tracking training code MIT, checkpoints not stated.
Verification: Confirmed