VLA Gallery / All models / Humanoid VLA

UH-1 (Humanoid-X)

USC·UC Berkeley·Toyota Research Institute · 2024-12

기타Humanoid VLANot discloseddiscrete tokens

A VLA-adjacent model that converts language into humanoid motion without vision input.

Paper

Learning from Massive Human Videos for Universal Humanoid Pose ControlarXiv 2412.14172

Overview

Category

Humanoid VLA

Type

기타

Released

2024-12

Organization

USC·UC Berkeley·Toyota Research Institute

Open weights

Unverified

Parameters

Not disclosed (18 layers·16 heads·1,024-dim)

Architecture

Backbone LLM

None

Vision encoder

—

Action head

VQ-VAE action tokens (discrete tokens)

Chunk

Not disclosed

Control rate

50 fps

Architecture (original summary)

18-layer · 16-head · 1,024-dim Transformer + VQ-VAE action tokenizer (codebook 2048 × 512, downsample k=4); text→keypoints (goal-conditioned RL policy, closed-loop) or text→joint targets (open-loop, lower body by separate locomotion policy), action sequence 50 fps

Training

Pretraining accelerators

RTX 6000 Ada × 1

Pretraining duration

Full Humanoid-X: tokenizer about 40 hours + transformer about 400 hours (HumanoidML3D subset: about 8 hours + about 30 hours)

Pretraining steps & batch

Not disclosed

GPU/TPU-hours

440 (Converted · RTX 6000 Ada-hours (1 × (about 40 h + about 400 h), full Humanoid-X))

Fine-tuning

N/A; low-level RL policy uses Isaac Gym 6192 envs, PPO (γ 0.99, λ 0.95), GPU and time not disclosed

Data type

Human video / motion capture

Data scale

Humanoid-X 163.8K motion samples·240.3 hours of video·20.7M frames, vocabulary 3206 words

Main datasets

Humanoid-X (AIST, AMASS, Charades, EgoBody, GRAB, HAA500, HuMMan, IDEA400, Kinetics700, MotionX Video, YouTube; captions by Video-LLaMA)

Tasks & evaluation

Real Unitree H1-2, 12 language commands × 10 trials, 'almost 100%' in both modes (per-task 80–100% in Table 3)

Robot

Unitree H1-2 whole-body pose control

Inference

Inference device & speed

Not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Partial (RL low-level controller) · Transformer is supervised; keypoint-mode execution uses an IsaacGym PPO goal-conditioned RL policy (6,192 environments)

Base control

Fixed · On the real robot, text→action open-loop controls only upper-body pose; lower body uses a separate pretrained locomotion policy (no locomotion in evaluation tasks)

Notes

Key features

A VLA-adjacent model that converts language into humanoid motion without vision input. Trained on Humanoid-X, which retargets internet human video to humanoid motion.

Notes

A VLA-adjacent model without vision input, so type is 'Other'. Data·code not released at announcement ('will fully release in the future'); current status Unverified. The vocabulary '11,897' in the HTML extraction is an error; the PDF original says 3206.

Verification

Confirmed