VLA Gallery / All models / Humanoid whole-body controllers

GMT

GMT (General Motion Tracking)

UC San Diego·Simon Fraser University · 2025-06

VAHumanoid whole-body controllersNot disclosedRL policy

Adaptive sampling (motions over 10 s cut to 10 s and weighted by completion) and MoE let a single policy track diverse human motions.

Paper

Backbone LLM: None
Vision encoder: —
Action head: Joint targets (RL policy)
Chunk: Not disclosed
Control rate: 50 Hz
Architecture (original summary): Motion Mixture-of-Experts MLP (set of expert networks + gating network) + convolutional motion encoder (future frames → 128-dim latent); student trained with DAgger on 20-step proprioceptive history; sim 500 Hz·control 50 Hz

Pretraining accelerators: RTX 4090 × 1
Pretraining duration: Teacher about 3 days + student about 1 day
Pretraining steps & batch: 4096 parallel env (IsaacGym); steps not disclosed
GPU/TPU-hours: 96 (Converted · RTX 4090-hours (1 × (about 3 days + about 1 day) × 24 h))
Fine-tuning: N/A
Data type: Motion capture
Data scale: AMASS · LAFAN1 subset 8925 clips · 33.12 hours
Main datasets: AMASS, LAFAN1
Tasks & evaluation: MPKPE·MPJPE·velocity tracking error evaluated; general motion tracking on real Unitree G1
Robot: Unitree G1 (23 DoF, height 1.32 m)

Reinforcement learning: Used (RL controller training) · RL teacher (about 3 days) → DAgger student (about 1 day), single RTX 4090
Base control: Humanoid legged (RL policy drives joints directly) · Whole-body motion tracking policy follows reference motion and directly outputs 23-DoF joint targets (walking, running, kicking, side-stepping)

Key features: Adaptive sampling (motions over 10 s cut to 10 s and weighted by completion) and MoE let a single policy track diverse human motions. Trained on a single RTX 4090: teacher about 3 days, student about 1 day.
Notes: The '33 days'·'11 day'·'89,258 clips' in the HTML extraction are misreads; PDF values are correct. Pretrained checkpoints for MuJoCo sim2sim released.
Verification: Confirmed