VLA Gallery / All models / Autoregressive & latent-action VLA

OpenVLA

Stanford·UC Berkeley·TRI·Google DeepMind·Physical Intelligence·MIT · 2024-06

VLAAutoregressive & latent-action VLA7Bdiscrete tokens

An open 7B VLA trained on 970k OXE trajectories; ECoT·TraceVLA·OpenVLA-OFT·UniVLA and others build on this stack.

Paper

OpenVLA: An Open-Source Vision-Language-Action ModelarXiv 2406.09246

Overview

Category

Autoregressive & latent-action VLA

Type

VLA

Released

2024-06

Organization

Stanford·UC Berkeley·TRI·Google DeepMind·Physical Intelligence·MIT

Open weights

Yes

Parameters

7B (paper)/7.5B (README); Table 1 Full FT 7,188.1M; vision 600M + Llama 2 7B

Architecture

Backbone LLM

Llama 2 7B · Prismatic-7B

Vision encoder

SigLIP + DINOv2 (600M)

Action head

7 discrete tokens of 256 bins (discrete tokens)

Chunk

None (1 step)

Control rate

Not disclosed

Architecture (original summary)

Prismatic-7B (SigLIP+DINOv2 600M → 2-layer MLP → Llama 2 7B); 256-bin discrete tokens per action dimension (last 256 of vocabulary), no chunks, 224×224 input

Training

Pretraining accelerators

64 × A100

Pretraining duration

14 days (paper)/15 days (project page)

Pretraining steps & batch

27 epochs, batch 2048, fixed lr 2e-5 (public checkpoint name is step-295000-epoch-40)

GPU/TPU-hours

21,500 (Stated · A100-hours)

Fine-tuning

LoRA r=32 (1.4% of parameters trained): single A100 10–15 h (minimum about 27 GB; batch 16 is 59.7 GB (paper)/about 72 GB (README)); full FT: 8 A100 (FSDP) 5–15 h per task

Data type

Real-robot teleoperation

Data scale

970k trajectories

Main datasets

OXE 'Open-X Magic Soup++' (Fractal 12.7%, Kuka 12.7%, Bridge 13.3%, DROID 10.0%, etc.); DROID excluded in the last 1/3 of training

Tasks & evaluation

BridgeData V2 WidowX · Google robot generalization evaluation; Franka-Tabletop · Franka-DROID fine-tuning (10–150 demos); LIBERO average 76.5% (84.7/88.4/79.2/53.7)

Robot

WidowX, Google robot, Franka

Inference

Inference device & speed

RTX 4090 about 6 Hz, bf16 15 GB; int8 10.2 GB·A5000 1.2 Hz; int4 7.0 GB·A5000 3 Hz; A100 4.2 Hz (secondary, measured by OpenVLA-OFT)

Inference memory

bf16 15 GB · int8 10.2 GB · int4 7.0 GB (paper)

Weights bf16 (calc.)

14 GB

Where it runs

Local PC

Reinforcement learning & base

Reinforcement learning

Not used · Imitation learning by action token prediction

Base control

Fixed · No base control (arm only: 7-dim EE displacement·gripper)

Notes

Key features

An open 7B VLA trained on 970k OXE trajectories; ECoT·TraceVLA·OpenVLA-OFT·UniVLA and others build on this stack. Pretraining compute stated as 21,500 A100-hours.

Notes

Training duration differs: 14 days in the paper, 15 days on the project page. The Full FT 163.3 GB in Table 1 is the total for batch 16 split across 2 GPUs with FSDP. The LIBERO fine-tuning appendix (App E) could not be checked. Parameters (B) is the paper value 7B.

Verification

Confirmed