VLA Gallery / All models / Flow·Diffusion VLA
Dita
Shanghai AI Lab, Zhejiang University, CUHK MMLab, Peking University, SenseTime Research, Tsinghua University, HKISI/CAS · 2024-10†
VLAFlow·Diffusion VLA334Mdiffusion
Performs diffusion denoising with in-context conditioning that puts language·image·timestep·noisy action tokens in one causal transformer sequence.
Paper
Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action PolicyarXiv 2503.19757
Overview
- Category
- Flow·Diffusion VLA
- Type
- VLA
- Released
- 2024-10†
- Organization
- Shanghai AI Lab, Zhejiang University, CUHK MMLab, Peking University, SenseTime Research, Tsinghua University, HKISI/CAS
- Open weights
- Yes
- Parameters
- 334M (221M trainable); about 11M trained with LoRA
Architecture
- Backbone LLM
- None (CLIP text encoder)
- Vision encoder
- DINOv2 + Q-Former
- Action head
- in-context diffusion denoising (DDIM 10) (diffusion)
- Chunk
- 16 steps
- Control rate
- 3 Hz (real robot)
- Architecture (original summary)
- No LLM: frozen CLIP text + DINOv2 + Q-Former (depth 4) + causal transformer (12 blocks, hidden 768); in-context diffusion denoising (DDPM 1000 for training, DDIM 20→10 steps), 2-frame observation→16-step chunk, 3 Hz on real robot
Training
- Pretraining accelerators
- A100×32 (4 nodes × 8)
- Pretraining duration
- Not disclosed
- Pretraining steps & batch
- 100,000 steps, batch 8192 (256 per GPU), AdamW, lr 1e-4 (transformer · Q-Former)/1e-5 (DINOv2)
- GPU/TPU-hours
- Not disclosed
- Fine-tuning
- LIBERO A100×8·batch 512·100K steps; CALVIN A100×4·batch 128·15 epochs; ManiSkill2 A100×16·batch 1024·50K steps; Franka 10-shot LoRA 20K steps·batch 512 (GPU not disclosed); all times not disclosed
- Data type
- Real-robot teleoperation + simulation (40K+ self-generated ManiSkill2 trajectories, for fine-tuning·evaluation)
- Data scale
- OXE (oxe_magic_soup_plus, trajectory count not disclosed); 10 samples per real-robot task
- Main datasets
- Open X-Embodiment, DROID (separate pretrained version)
- Tasks & evaluation
- SimplerEnv coke can 0.837/0.855, move near 0.760/0.730, drawer 0.463/0.375; LIBERO 84.2/96.3/85.4/63.8, average 82.4%; CALVIN ABC→D 3.61; ManiSkill2; Franka 10-shot real robot (2-stage task 63.8%)
- Robot
- Franka (single arm); simulation
Inference
- Inference device & speed
- Deployment server 1× A100; latency Not disclosed
- Inference memory
- Not disclosed
- Weights bf16 (calc.)
- 0.7 GB
- Where it runs
- Remote server (A100)
Reinforcement learning & base
- Reinforcement learning
- Not used · Imitation learning (behavior cloning) only
- Base control
- Fixed · No base control (Franka single arm)
Notes
- Key features
- Performs diffusion denoising with in-context conditioning that puts language·image·timestep·noisy action tokens in one causal transformer sequence. At 334M, after OXE pretraining it adapted on Franka with 10 samples per task.
- Notes
- Release month based on the earlier version arXiv 2410.15959 (submission-date line could not be confirmed†); this version 2503.19757 v1 is 2025-03-25. Affiliation corrected from the notes' RoboDita team to page 1 of the paper. ManiSkill2 per-task figures excluded due to extraction mismatch.
- Verification
- Corrections applied