VLA GalleryAll modelsGallery

VLA Gallery / All models / Autoregressive & latent-action VLA

villa-X

Microsoft Research·Tsinghua University·Wuhan University·HKUST·Nanjing University · 2025-07

VLAAutoregressive & latent-action VLALAM 955Mflow matching

Uses latent actions as an intermediate plan, trained jointly with a robot action expert.

View architecture diagram in the galleryPaper ↗

Paper

villa-X: Enhancing Latent Action Modeling in Vision-Language-Action ModelsarXiv 2507.23682

Overview

Category
Autoregressive & latent-action VLA
Type
VLA
Released
2025-07
Organization
Microsoft Research·Tsinghua University·Wuhan University·HKUST·Nanjing University
Open weights
Partial
Parameters
LAM 955M (README); actor not disclosed

Architecture

Backbone LLM
Not disclosed
Vision encoder
Not disclosed
Action head
Latent action expert + flow matching robot action expert (flow matching)
Chunk
Not disclosed
Control rate
Not disclosed
Architecture (original summary)
Latent action model (LAM) + actor (latent action expert, robot action expert); robot actions generated with flow matching

Training

Pretraining accelerators
LAM: 128 A100; actor: 64 A100
Pretraining duration
LAM about 4 days; actor 4 days
Pretraining steps & batch
Not disclosed
GPU/TPU-hours
6,144 (Converted · A100-hours (actor 64 × 96 h; LAM 128 × about 96 h = 12,288 separately))
Fine-tuning
Not disclosed
Data type
Real-robot teleoperation + human video
Data scale
Robot 1.6M trajectories (223.5M frames) + human video 3.6M clips
Main datasets
Fractal, BridgeData V2, Something-Something V2, etc. (LAM mix ratio Fractal 10% · Bridge V2 10% · SSv2 100%)
Tasks & evaluation
SIMPLER Google Robot 77.7%·WidowX 62.5%; real Realman gripper·XArm hand
Robot
Google Robot·WidowX (SIMPLER), Realman arm (gripper), XArm (hand)

Inference

Inference device & speed
Not disclosed
Inference memory
Not disclosed
Weights bf16 (calc.)
Not disclosed
Where it runs
Not disclosed

Reinforcement learning & base

Reinforcement learning
Not used · Latent action pretraining + imitation learning
Base control
Fixed · No base control (arm·hand)

Notes

Key features
Uses latent actions as an intermediate plan, trained jointly with a robot action expert. Pretrained on 1.6M robot trajectories and 3.6M human video clips.
Notes
The notes' 'compute Not disclosed' and 'data ratios only' were corrected in verification. GPU-hours cover actor pretraining only; adding the LAM gives about 18,432 A100-hours. Only the LAM weights are released.
Verification
Corrections applied

Other models in this category