VLA Gallery / All models / Humanoid whole-body controllers

VIRAL

VIRAL (Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation)

NVIDIA·CMU·UC Berkeley·CUHK · 2025-11

VAHumanoid whole-body controllersNot disclosedRL policy

Distills an RL teacher into an RGB student in large-scale tiled-rendering sim, transferring vision-based loco-manipulation to the real robot.

Paper

VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-ManipulationarXiv 2511.15200

Overview

Category

Humanoid whole-body controllers

Type

VA

Released

2025-11

Organization

NVIDIA·CMU·UC Berkeley·CUHK

Open weights

Unverified

Parameters

Not disclosed

Architecture

Backbone LLM

None

Vision encoder

DINOv3 (student)

Action head

WBC command deltas 11-dim (RL policy)

Chunk

Not disclosed

Control rate

Not disclosed

Architecture (original summary)

Teacher: MLP [512, 256, 128], privileged state → 11-dim WBC command delta (Δv, Δω_yaw, Δq_arm, Δq_finger) / Student: 108×192 RGB encoded by DINOv3 (ViT) into 128-dim features combined with state, MLP·feedforward history·LSTM head

Training

Pretraining accelerators

Teacher L40S × 16 (2 8-GPU nodes), student L40S × 64 (8 8-GPU nodes)

Pretraining duration

Not disclosed

Pretraining steps & batch

env: teacher 32768 (2048*8GPUs*2Nodes), student 65535 (1024*8GPUs*8Nodes); steps not disclosed

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A

Data type

Simulation·synthetic

Data scale

200 sim teleoperation demonstrations (reference state initialization); the rest is sim RL·distillation

Main datasets

In-house sim teleoperation demonstrations

Tasks & evaluation

54 successes in 59 consecutive real trials; cycle 20.2 s (expert teleoperation 100%·21.4 s)

Robot

Unitree G1 (29-DoF) + 7-DoF three-finger hands

Inference

Inference device & speed

Not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Used (RL controller training) · RL teacher (privileged state) → RGB student distillation. Initial states set from 200 sim teleoperation demos

Base control

Humanoid legged (Commands/latents → RL locomotion controller) · Policy outputs WBC command deltas (Δv, Δω_yaw, arms, fingers) → HOMIE WBC handles lower-body locomotion (walking between tables, turning)

Notes

Key features

Distills an RL teacher into an RGB student in large-scale tiled-rendering sim, transferring vision-based loco-manipulation to the real robot. Student training uses 64 L40S.

Notes

Isaac Lab tiled rendering. Env count kept as in the original's parentheses. Wall-clock not disclosed (verification NOT FOUND). Student visual encoder DINOv3 is a ViT, but the policy head is MLP·LSTM, so classified as non-Transformer for comparison.

Verification

Confirmed