VLA Gallery / All models / Flow·Diffusion VLA
DexVLA
Midea Group, East China Normal University · 2025-02
VLAFlow·Diffusion VLAVLM 2B + diffusion expert 1Bdiffusion
Uses a 3-stage curriculum that first pretrains a 1B diffusion expert on cross-embodiment data, and adds sub-step reasoning annotations about every 5 seconds.
Paper
DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot ControlarXiv 2502.05855
Overview
- Category
- Flow·Diffusion VLA
- Type
- VLA
- Released
- 2025-02
- Organization
- Midea Group, East China Normal University
- Open weights
- Partial
- Parameters
- VLM 2B + diffusion expert 1B (README: Qwen2-VL ~2B, ScaleDP-H ~1B, ScaleDP-L ~400M); total not stated
Architecture
- Backbone LLM
- Qwen2-1.5B · Qwen2-VL-2B config
- Vision encoder
- Qwen2-VL ViT
- Action head
- ScaleDP diffusion expert 1B (32 layers) (diffusion)
- Chunk
- Not disclosed
- Control rate
- Not disclosed
- Architecture (original summary)
- Qwen2-VL-2B + 1B ScaleDP diffusion expert (32 layers, hidden 1280, 16 heads) + per-embodiment MLP heads; chunk length not disclosed
Training
- Pretraining accelerators
- H100×8(Stage 1)
- Pretraining duration
- Not disclosed
- Pretraining steps & batch
- Stage 1 5 epochs, lr 1e-4 constant, AdamW (β 0.9/0.95); batch not disclosed
- GPU/TPU-hours
- Not disclosed
- Fine-tuning
- Stage 2·3 each 5 epochs (lr 2e-5), GPU·time not disclosed; README: on 1 A6000 with batch 2, default 38 GB·1.1 s/iter, small ScaleDP 33 GB, LoRA VLM 24 GB, DeepSpeed offload 20~29 GB·6.56 s/iter
- Data type
- Real-robot teleoperation + sub-step language annotations
- Data scale
- 100 h, 91 tasks (AgileX 42.7%, single Franka 34.7%, etc.); at most 100 demos per task for new embodiments
- Main datasets
- In-house collected data
- Tasks & evaluation
- Shirt folding 0.92 (bimanual AgileX), drink pouring 0.90 (Franka+dexterous hand), packing 0.90 (bimanual UR5e), dryer unloading 0.8, laundry folding, Table bussing, bin picking, sorting
- Robot
- Franka + gripper, Franka + dexterous hand, bimanual UR5e, bimanual AgileX
Inference
- Inference device & speed
- 1 × A6000 60 Hz
- Inference memory
- Not disclosed
- Weights bf16 (calc.)
- Not disclosed
- Where it runs
- Local PC
Reinforcement learning & base
- Reinforcement learning
- Not used · Imitation learning (behavior cloning) only
- Base control
- Fixed · No base control (AgileX 14-dim joint states·actions, mobile chassis not used)
Notes
- Key features
- Uses a 3-stage curriculum that first pretrains a 1B diffusion expert on cross-embodiment data, and adds sub-step reasoning annotations about every 5 seconds. Learned new embodiments with 100 or fewer demos.
- Notes
- Only Stage 1 ScaleDP-H·ScaleDP-L weights released. Speedup when training only the diffusion expert is inconsistent within the source ('5×' vs '2.78×'). Parameters (B) null since total not stated.
- Verification
- Confirmed