VLA Gallery / All models / Autoregressive & latent-action VLA

RoboMamba

Peking University·AI²Robotics·BAAI · 2024-06

VLAAutoregressive & latent-action VLAMamba 2.8B/1.4B + ViT-Lregression

VLA that uses a Mamba state-space model as the LLM instead of a transformer.

Paper

Backbone LLM: Mamba 2.8B (SSM)
Vision encoder: CLIP/SigLIP ViT-L
Action head: MLP pose regression (3.7M) (regression)
Chunk: Not disclosed
Control rate: Not disclosed
Architecture (original summary): Mamba SSM LLM 2.8B/1.4B + CLIP/SigLIP ViT-Large; continuous SE(3) end-effector pose via MLP pose regression head (3.7M), no action tokens

Pretraining accelerators: NVIDIA A100 (count not disclosed)
Pretraining duration: Not disclosed
Pretraining steps & batch: Alignment 1 epoch (LLaVA-LCS 558K); co-training 2 epochs (LLaVA mixture 655K + RoboVQA 300K)
GPU/TPU-hours: Not disclosed
Fine-tuning: A100 (count not disclosed), 8 epochs (10K SAPIEN images), 'a few dozen minutes'
Data type: Web vision-language + robot VQA + simulation·synthetic
Data scale: LLaVA-LCS 558K + LLaVA mix 655K + RoboVQA 300K; SAPIEN 10K images
Main datasets: LLaVA-LCS 558K, LLaVA mix, RoboVQA, SAPIEN
Tasks & evaluation: SAPIEN manipulation seen 63.3%·unseen 53%; RoboVQA
Robot: SAPIEN simulation, real robot (model not stated)

Inference device & speed: 3× faster than prior VLAs (absolute Hz Not stated)
Inference memory: Not disclosed
Weights bf16 (calc.): 5.6 GB
Where it runs: Not disclosed

Reinforcement learning: Not used · Imitation learning (behavior cloning) only
Base control: Fixed · No base control (arm EE pose regression)

Key features: VLA that uses a Mamba state-space model as the LLM instead of a transformer. Manipulation fine-tuning takes tens of minutes, and it reports 3× faster inference than existing VLAs.
Notes: A reference row for non-transformer comparison. Parameters (B) refer to the largest Mamba LLM (2.8B), excluding ViT-L. Whether the repository·weights are released could not be confirmed†.
Verification: Confirmed