Cosmos Policy

NVIDIA + Stanford · 2026-01

기타New in 20262Bvideo world model

A video model policy that trains a pretrained video model as a policy without architectural changes.

Paper

Backbone LLM: None (video diffusion backbone)
Vision encoder: —
Action head: Generates action·future state·value as latent frames (video world model)
Chunk: 50 steps (2 s)
Control rate: 25 Hz
Architecture (original summary): Cosmos-Predict2-2B-Video2World video diffusion model (no architectural changes); generates actions · future states · values together as latent frames; ALOHA chunk 50 steps (2 s), control 50→25 Hz

Pretraining accelerators: N/A (initialized from a video model, video pretraining compute not reported)
Pretraining duration: N/A
Pretraining steps & batch: N/A
GPU/TPU-hours: Not disclosed
Fine-tuning: LIBERO 64 H100 48 hours (40K steps·batch 1920); RoboCasa 32 H100 48 hours (45K·800); ALOHA 8 H100 48 hours (50K·200)
Data type: Simulation·synthetic (LIBERO·RoboCasa human teleoperation demonstrations) + real-robot teleoperation (ALOHA)
Data scale: LIBERO 500 demos† (paper)/2000 (README); RoboCasa 50 per task (24 tasks, README 1200); ALOHA 15–80 per task (README under 200)
Main datasets: LIBERO, RoboCasa, ALOHA
Tasks & evaluation: LIBERO average 98.5%; RoboCasa 67.1% (50 demos per task, previously 300); real ALOHA average score 93.6
Robot: Simulation single arm (LIBERO·RoboCasa), real bimanual ALOHA

Inference device & speed: Single H100 0.61 s (5 denoising steps), ALOHA 0.95 s (10 steps); planning search on 8 H100 4.9 s
Inference memory: Not disclosed
Weights bf16 (calc.): 4 GB
Where it runs: Not disclosed

Reinforcement learning: Partial (Value learning/planning) · Policy is supervised. Value prediction is learned from Monte Carlo returns of rollouts and used for best-of-N model-based planning (the paper does not call it RL)
RL evidence: https://arxiv.org/html/2601.16163v1
Base control: Fixed · No base control (LIBERO·RoboCasa sim arms, real ALOHA fixed bimanual)

Key features: A video model policy that trains a pretrained video model as a policy without architectural changes. Supports best-of-N model-based planning.
Notes: Corrections applied: code·model·training data released (e.g. nvidia/Cosmos-Policy-LIBERO-Predict2-2B). The note classifies it as a VLA-class video model policy (type is Other since it jointly generates video·actions). Comparison models π0.5·π0·OpenVLA-OFT+ also used the same budget of 8 H100s×48 h. Whether it is a DiT is not restated in the sources.
Verification: Corrections applied