VLA Gallery / All models / Flow·Diffusion VLA
DexGraspVLA
Peking University Institute for AI, PKU-PsiBot Joint Lab, HKUST(GZ) · 2025-02
VLAFlow·Diffusion VLAcontroller trainable 163Mdiffusion
An off-the-shelf VLM planner specifies the target object, and a DiT controller taking segmentation masks and DINOv2 features performs dexterous-hand grasping.
Paper
DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous GraspingarXiv 2502.20900
Overview
- Category
- Flow·Diffusion VLA
- Type
- VLA
- Released
- 2025-02
- Organization
- Peking University Institute for AI, PKU-PsiBot Joint Lab, HKUST(GZ)
- Open weights
- Yes
- Parameters
- controller trainable 163M; planner Qwen2.5-VL-72B (off-the-shelf model)
Architecture
- Backbone LLM
- Qwen2.5-72B · Qwen2.5-VL-72B config (planner)
- Vision encoder
- SAM · Cutie · DINOv2 × 2
- Action head
- DiT diffusion controller (16 steps) (diffusion)
- Chunk
- 64 steps
- Control rate
- 20 Hz
- Architecture (original summary)
- Hierarchical: VLM planner (Qwen-VL-Chat; appendix·README say Qwen2.5-VL-72B-Instruct) + DiT diffusion controller (SAM·Cutie segmentation, 3 vision encoders including 2 frozen DINOv2); chunk H=64, 16 inference steps, 20 Hz
Training
- Pretraining accelerators
- N/A (no large-scale robot pretraining)
- Pretraining duration
- N/A
- Pretraining steps & batch
- N/A
- GPU/TPU-hours
- Not disclosed
- Fine-tuning
- controller training: A800×8 server, under 1 day, 84 epochs, batch 48 per GPU
- Data type
- Real-robot demonstrations (kinesthetic teaching)
- Data scale
- 2,094 successful episodes, 36 household objects
- Main datasets
- In-house collection
- Tasks & evaluation
- 90.8% success on 1,287 combinations of 360 unseen objects·3 lighting conditions·6 backgrounds
- Robot
- 7-DoF RealMan RM75-6F arm + 6-DoF PsiBot hand
Inference
- Inference device & speed
- Grasping averages about 6 s per object; planner (72B) served on an A800×8 server using 4 GPUs·vLLM·speculative decoding (Qwen2.5-VL-7B)
- Inference memory
- Not disclosed
- Weights bf16 (calc.)
- Not disclosed
- Where it runs
- Remote server (planner)
Reinforcement learning & base
- Reinforcement learning
- Not used · Imitation learning (behavior cloning) only
- Base control
- Fixed · No base control (RealMan arm + hand)
Notes
- Key features
- An off-the-shelf VLM planner specifies the target object, and a DiT controller taking segmentation masks and DINOv2 features performs dexterous-hand grasping. Achieved 90.8% success on 1,287 unseen combinations.
- Notes
- Parameters (B) is null since the system total is not stated. Hierarchical rather than a single end-to-end VLA; controller training is classified as task training. Planner serving cost exceeds controller training cost. Weights are the controller checkpoint (planner is an off-the-shelf VLM). Institution corrected from the notes' Psi-Robot etc. per page 1 of the paper.
- Verification
- Corrections applied