VLA Gallery / All models / Humanoid whole-body controllers

HOVER

NVIDIA·CMU·UC Berkeley·UT Austin·UC San Diego · 2024-10

VAHumanoid whole-body controllersNot disclosedRL policy

Unifies multiple control modes into one multi-mode whole-body policy via command modes·sparse masks.

Paper

HOVER: Versatile Neural Whole-Body Controller for Humanoid RobotsarXiv 2410.21229

Overview

Category

Humanoid whole-body controllers

Type

VA

Released

2024-10

Organization

NVIDIA·CMU·UC Berkeley·UT Austin·UC San Diego

Open weights

Unverified

Parameters

Not disclosed (MLP [512, 256, 128])

Architecture

Backbone LLM

None

Vision encoder

—

Action head

Joint targets (multi-mode mask) (RL policy)

Chunk

Not disclosed

Control rate

Not disclosed

Architecture (original summary)

3-layer MLP [512, 256, 128]; student takes last 25 steps of proprioception + mode · sparse mask (sampled B(0.5)) → joint targets; DAgger distillation from oracle teacher

Training

Pretraining accelerators

One of RTX 4090 / RTX A6000 / L40 (repo, Isaac Lab 2.0.0 reimplementation); paper uses IsaacGym, GPU not disclosed

Pretraining duration

Teacher 100,000 iterations: RTX 4090 23.3 h / A6000 52.8 h / L40 44.6 h; student 10,000 iterations: 0.27 / 0.50 / 0.49 h (repo)

Pretraining steps & batch

Teacher 100,000 iterations + student 10,000 iterations (repo table); 50k~80k iterations usually suffice, example 1,024 envs

GPU/TPU-hours

24 (Converted · RTX 4090-hours (1 × (teacher 23.3 h + student 0.27 h), based on repo reimplementation))

Fine-tuning

N/A

Data type

Motion capture

Data scale

H1-retargeted AMASS (subset size not disclosed); full retargeting takes up to 4 days on a 32-core CPU

Main datasets

AMASS

Tasks & evaluation

Evaluated on tracking error and mode switching; 20 standing motion sequences on hardware

Robot

Unitree H1 (19-DOF, about 51.5 kg, about 1.8 m)

Inference

Inference device & speed

Control rate not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Used (RL controller training) · Oracle RL teacher → DAgger student distillation

Base control

Humanoid legged (RL policy drives joints directly) · Multi-mode commands (root velocity·height·pose, keypoints, joint angles) → whole-body 19 DoF joint target output; real-world forward walking demonstrated

Notes

Key features

Unifies multiple control modes into one multi-mode whole-body policy via command modes·sparse masks. The repo reimplementation trains the teacher in about a day on a single GPU.

Notes

Training time is from the repo (Isaac Lab 2.0.0 reimplementation), not the paper. Code Apache 2.0, checkpoints unverified. Verification covered only repo values (training time, envs, iterations).

Verification

Confirmed