VLA Gallery / All models / Humanoid whole-body controllers

HumanPlus HST

HumanPlus HST (Humanoid Shadowing Transformer)

Stanford University · 2024-06

VAHumanoid whole-body controllersNot disclosedRL policy

A low-level tracker for shadowing that transfers human motion to a humanoid in real time.

Paper

HumanPlus: Humanoid Shadowing and Imitation from HumansarXiv 2406.10454

Overview

Category

Humanoid whole-body controllers

Type

VA

Released

2024-06

Organization

Stanford University

Open weights

Unverified

Parameters

Not disclosed (repo default 4 layers·128-dim·4 heads)

Architecture

Backbone LLM

None

Vision encoder

—

Action head

Joint position targets (19-dim) (RL policy)

Chunk

Not disclosed

Control rate

50 Hz

Architecture (original summary)

Decoder-only transformer (repo default 4 layers·latent 128·4 heads, dropout 0.1), context 8 steps; proprioception + target pose → 19-dim joint position targets, 50 Hz

Training

Pretraining accelerators

Not disclosed (Isaac Gym PPO, repo)

Pretraining duration

Not disclosed

Pretraining steps & batch

max_iterations 15000 · 32 steps per env, 2048 envs (repo H1 config)

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A

Data type

Motion capture

Data scale

AMASS 40 hours, over 11,000 sequences†

Main datasets

AMASS

Tasks & evaluation

Real-time whole-body shadowing (teleoperation); autonomous tasks combined with HIT: Fold Clothes 100% (40 demos), Warehouse 90%, Wear a Shoe and Walk 60%

Robot

33-DoF humanoid based on Unitree H1 + Inspire-Robots RH56DFX hands, height 180cm

Inference

Inference device & speed

50 Hz; when shadowing, human pose estimation · retargeting at 25 fps on RTX 4090, hand pose estimation 10 fps

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Used (RL controller training) · Shadowing transformer trained with Isaac Gym PPO (2,048 envs, 15,000 iterations)

Base control

Humanoid legged (RL policy drives joints directly) · HST takes target forward/lateral·yaw velocity·joint targets and directly outputs 19-dim joint positions including legs (walking·squatting)

Notes

Key features

A low-level tracker for shadowing that transfers human motion to a humanoid in real time. It is the lower layer of a two-level structure in which the upper HIT sends target poses asynchronously at 25 Hz.

Notes

'Over 11,000 sequences' could not be reconfirmed in verification extraction, hence †. The paper body gives no simulator, GPU, time or parameters (Isaac Gym, env count and iteration count are in the repo). Code and data released; checkpoints unverified.

Verification

Confirmed