VLA Gallery / All models / Autoregressive & latent-action VLA

UniVLA (OpenDriveLab)

HKU·OpenDriveLab·AgiBot · 2025-05

VLAAutoregressive & latent-action VLA7B + action decoder 10.8Mlatent actions

Learns latent actions in two stages, task-agnostic and task-centric, so even human video and navigation data can be used for pretraining.

Paper

UniVLA: Learning to Act Anywhere with Task-centric Latent ActionsarXiv 2505.06111

Overview

Category

Autoregressive & latent-action VLA

Type

VLA

Released

2025-05

Organization

HKU·OpenDriveLab·AgiBot

Open weights

Yes

Parameters

7B + action decoder 10.8M (abstract)/12.6M (body)/about 12M (README); about 123M trainable parameters with LoRA

Architecture

Backbone LLM

Llama 2 7B · Prismatic-7B

Vision encoder

SigLIP + DINOv2 (+ DINOv2 latent actions)

Action head

Latent action tokens → per-embodiment decoder (latent actions)

Chunk

12 (real robot)

Control rate

Not disclosed

Architecture (original summary)

Prismatic-7B; task-centric VQ-VAE latent actions on DINOv2 features (codebook 16, 4 tokens) + per-embodiment action decoders, real-robot chunk 12

Training

Pretraining accelerators

32 × A100

Pretraining duration

About 30 h

Pretraining steps & batch

20,000 steps, batch 1,024 (32 per GPU), lr 2e-5

GPU/TPU-hours

960 (Stated · A100-hours)

Fine-tuning

LoRA r=32, 8 GPUs (README command); LIBERO 30k–40k steps·CALVIN 100k steps, batch 128; time not disclosed

Data type

Real-robot teleoperation (manipulation) + real-robot navigation (GNM) + human video

Data scale

Not disclosed (mixture ratio only: Fractal 13.9%, Kuka 6.3%, Bridge 6.8%, Ego4D 3.0%, etc.)

Main datasets

OXE subset, GNM, Ego4D

Tasks & evaluation

LIBERO average 95.2% (Bridge only 92.5%); CALVIN; SimplerEnv-Bridge; R2R oracle success 47.1%; real AgileX 68.9%

Robot

AgileX, LIBERO · CALVIN · SimplerEnv simulation, R2R navigation

Inference

Inference device & speed

RTX 4090 10 Hz (closed loop)

Inference memory

Not disclosed

Weights bf16 (calc.)

14 GB

Where it runs

Local PC

Reinforcement learning & base

Reinforcement learning

Not used · Latent action pretraining + imitation learning fine-tuning

Base control

Sim only (Discrete motion actions) · Real robot is a fixed Piper arm; for navigation, latent actions are decoded into discrete motion actions for an R2R VLN-CE sim agent

Notes

Key features

Learns latent actions in two stages, task-agnostic and task-centric, so even human video and navigation data can be used for pretraining. Pretraining took 960 A100-hours, about 5% of OpenVLA.

Notes

The Bridge-only and human-video-only variants use 8 GPUs, global batch 258 (as written in the original), about 200 A100-hours. The latent action model uses batch 512 · 100k steps; time not disclosed. Domains also include navigation (R2R). A different model from BAAI's UniVLA.

Verification

Confirmed