VLA Gallery / All models / Flow·Diffusion VLA

RDT2

RDT2 (RDT2-FM / RDT2-VQ)

Tsinghua University (thu-ml) · 2025-09

VLAFlow·Diffusion VLATotal not disclosedflow matching

Trained on about 10,000 hours of human demonstrations collected in 100+ homes with an improved UMI handheld device instead of robot teleoperation.

Paper

Category: Flow·Diffusion VLA
Type: VLA
Released: 2025-09
Organization: Tsinghua University (thu-ml)
Open weights: Yes
Parameters: Total not disclosed; Qwen backbone about 7B (hidden 3584) + action expert about 400M (hidden 1024)

Backbone LLM: Qwen2.5-7B · Qwen2.5-VL-7B config
Vision encoder: Qwen2.5-VL ViT
Action head: flow expert about 400M (FM) / RVQ tokens (VQ) (flow matching)
Chunk: 24 (README) / 32 (paper)
Control rate: 30 Hz
Architecture (original summary): Qwen2.5-VL-7B-Instruct + about 400M RDT-family flow matching action expert (RDT2-FM); RDT2-VQ autoregresses Residual VQ tokens; chunk 24 steps (README, 0.8 s at 30 fps)/32 steps (paper deployment), data 30 Hz

Pretraining accelerators: GPU model not disclosed (App. D.5: 7 nodes × 8 GPUs)
Pretraining duration: Not disclosed
Pretraining steps & batch: RDT2-VQ 128K iter, RDT2-FM 66K iter, batch 96 per GPU, lr 1e-4 (App. D.5: global 5,376 / 2,304)
GPU/TPU-hours: Not disclosed
Fine-tuning: 1 node × 8 GPUs (model not disclosed), 50K steps, global batch 96; UltraFast distillation 20K steps on the same hardware; README: RDT2-FM expert fine-tuning ~16 GB (RTX 4090), RDT2-VQ LoRA >32 GB (A100 40GB), Full >80 GB (A100 80GB/H100/B200)
Data type: Human video (UMI handheld demonstrations) + web vision-language
Data scale: UMI demonstrations 10,000+ h (abstract·body)/about 10,000 h (Appendix B.1), 100+ homes; VL QA 12M+ pairs
Main datasets: In-house UMI data; Ego4D, HD-EPIC, RoboVQA, RoboBrain (QA)
Tasks & evaluation: Zero-shot 4U: Pick, Pick & Place, Wiping, Shaking, Button Pressing; fine-tuning: clothes folding, Table Bussing, Unzipping, dynamic button pressing, table tennis (256 trials)
Robot: UMI gripper interface (embodiment-agnostic); validated on bimanual UR5e, bimanual Franka FR3

Inference device & speed: README inference ~16 GB VRAM (RTX 4090); ms·Hz figures not disclosed
Inference memory: About 16 GB (RTX 4090, README)
Weights bf16 (calc.): Not disclosed
Where it runs: Not disclosed

Reinforcement learning: Not used · Imitation learning from human UMI demonstrations
Base control: Fixed · No base control (UMI gripper interface, UR5e·Franka fixed bimanual)

Key features: Trained on about 10,000 hours of human demonstrations collected in 100+ homes with an improved UMI handheld device instead of robot teleoperation. Presented zero-shot execution on unseen robots·scenes·objects·instructions (4U) and UltraFast, a 1-step distilled version.
Notes: Model released 2025-09 (README), paper arXiv 2026-02-03. The App. D.5 sentence is in the ablation section, so it is unclear whether it refers to the main training. Flow scope is RDT2-FM; RDT2-VQ is autoregressive. Chunk 24 (README)/32 (paper) are both original values.
Verification: Confirmed