VLA Gallery / All models / Flow·Diffusion VLA
RDT-1B
Tsinghua University (thu-ml) · 2024-10
VLAFlow·Diffusion VLA1.2Bdiffusion
Diffusion policy trained jointly on 46 heterogeneous robot datasets through a unified action space.
Paper
RDT-1B: a Diffusion Foundation Model for Bimanual ManipulationarXiv 2410.07864
Overview
- Category
- Flow·Diffusion VLA
- Type
- VLA
- Released
- 2024-10
- Organization
- Tsinghua University (thu-ml)
- Open weights
- Yes
- Parameters
- 1.2B (paper)/1B (README · model card); small RDT-170M
Architecture
- Backbone LLM
- None (T5-XXL text encoder)
- Vision encoder
- SigLIP so400m (frozen)
- Action head
- Diffusion Transformer 1.2B (DPM-Solver++ 5 steps) (diffusion)
- Chunk
- 64 steps
- Control rate
- 25 Hz (README example)
- Architecture (original summary)
- No LLM: frozen SigLIP (so400m) + frozen T5-XXL + Diffusion Transformer; diffusion continuous action chunk of 64 steps, inference DPM-Solver++ 5 steps (reduced from 100), README example 25 Hz
Training
- Pretraining accelerators
- H100 80GB×48
- Pretraining duration
- About 1 month
- Pretraining steps & batch
- 1M steps / batch unverified in paper† (pretrain.sh default 32, lr 1e-4, bf16)
- GPU/TPU-hours
- 34,560 (Converted (approx.) · H100-hours (48 × about 1 month = 30 days))
- Fine-tuning
- H100 80GB×48, 3 days, 130K steps (ALOHA 6K+ episodes); finetune.sh default batch 32·200K steps, README recommends 150K+ steps
- Data type
- Real-robot teleoperation
- Data scale
- Pretraining 46 datasets 1M+ trajectories·21TB; fine-tuning 6K+ episodes·300+ tasks·15+ rooms
- Main datasets
- 46 datasets including RT-1 (130K), DROID (76K), RH20T (110K); in-house Mobile ALOHA data
- Tasks & evaluation
- 7 real ALOHA bimanual tasks (Wash Cup, Pour Water, Pour Water-L-1/3, Pour Water-R-2/3, Handover, Fold Shorts, Robot Dog); ManiSkill (5,000 trajectories, 300k iter fine-tuning); RoboTwin 50 tasks
- Robot
- Mainly ALOHA (Mobile ALOHA) bimanual, single- and bimanual cross-embodiment
Inference
- Inference device & speed
- Onboard RTX 4090 24GB: chunks 6 Hz, average actions 381 Hz
- Inference memory
- Not disclosed
- Weights bf16 (calc.)
- 2.4 GB
- Where it runs
- Onboard (RTX 4090)
Reinforcement learning & base
- Reinforcement learning
- Not used · Imitation learning (behavior cloning) only
- Base control
- Fixed · The unified 128-dim action space has base velocity slots, but evaluation (Mobile ALOHA bimanual tasks) does not mention base motion
Notes
- Key features
- Diffusion policy trained jointly on 46 heterogeneous robot datasets through a unified action space. Fine-tuned on 6K+ bimanual ALOHA episodes.
- Notes
- Parameters inconsistent across sources, 1.2B (paper)/1B (README); parameters (B) use the paper value. Batch size and learning rate in the paper could not be confirmed in verification. ICLR 2025 poster.
- Verification
- Confirmed