VLA Gallery / All models / Humanoid whole-body controllers

HumanPlus HIT

HumanPlus HIT (Humanoid Imitation Transformer)

Stanford University · 2024-06

VAHumanoid whole-body controllersNot disclosedregression

A visuomotor policy that learns autonomous whole-body skills from up to 40 demonstrations per task collected via shadowing.

Paper

HumanPlus: Humanoid Shadowing and Imitation from HumansarXiv 2406.10454

Overview

Category

Humanoid whole-body controllers

Type

VA

Released

2024-06

Organization

Stanford University

Open weights

Unverified

Parameters

Not disclosed

Architecture

Backbone LLM

None

Vision encoder

ResNet18 (stereo RGB)

Action head

50-chunk target poses → HST (regression)

Chunk

50 steps

Control rate

25 Hz

Architecture (original summary)

Transformer (repo example hidden 512, FF 512, 6 decoder layers, ResNet18 backbone); stereo egocentric RGB features + proprioception + fixed positional embeddings → chunk of 50 target poses, sent asynchronously to HST onboard at 25 Hz

Training

Pretraining accelerators

Not disclosed

Pretraining duration

Not disclosed

Pretraining steps & batch

repo example 100,000 steps · batch 48 (lr 1e-5)

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A

Data type

Real-robot teleoperation

Data scale

Up to 40 demonstrations per task (shadowing collection)

Main datasets

In-house shadowing demonstrations

Tasks & evaluation

Fold Clothes 100% (40 demos), Rearrange Objects 90% (30), Type AI 80% (30), Two-Robot Greeting 90% (30), Warehouse 90% (25), Wear a Shoe and Walk 60% (40)

Robot

33-DoF humanoid based on Unitree H1 + Inspire-Robots RH56DFX hands, height 180cm

Inference

Inference device & speed

Onboard 25 Hz

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Onboard

Reinforcement learning & base

Reinforcement learning

Partial (RL low-level controller) · HIT uses imitation learning on teleoperation data; execution is handled by RL-trained HST

Base control

Humanoid legged (Commands/latents → RL locomotion controller) · HIT sends target pose chunks asynchronously to HST → HST executes whole-body joints including legs (forward walking in autonomous 'walk wearing shoes')

Notes

Key features

A visuomotor policy that learns autonomous whole-body skills from up to 40 demonstrations per task collected via shadowing. Forms a two-level structure with the low-level HST.

Notes

--dec_layers 6·--backbone resnet18 from the repo example are NEW from verification. Training GPU·time not disclosed. Code·data released, checkpoint unverified.

Verification

Confirmed