VLA Gallery / All models / Autoregressive & latent-action VLA

MolmoAct

Allen Institute for AI·University of Washington · 2025-08

VLAAutoregressive & latent-action VLA7Bdiscrete tokens

An 'action reasoning model' that outputs actions via depth perception and visual trajectories.

Paper

MolmoAct: Action Reasoning Models that can Reason in SpacearXiv 2508.07917

Overview

Category

Autoregressive & latent-action VLA

Type

VLA

Released

2025-08

Organization

Allen Institute for AI·University of Washington

Open weights

Yes

Parameters

7B (both backbones)

Architecture

Backbone LLM

Qwen2.5-7B / OLMo2-7B

Vision encoder

SigLIP2 / OpenCLIP

Action head

Depth tokens → trajectory → 256-bin tokens (discrete tokens)

Chunk

8 steps

Control rate

Not disclosed

Architecture (original summary)

SigLIP2+Qwen2.5-7B (7B-D) or OpenCLIP+OLMo2-7B (7B-O); depth tokens (codebook 128, 100 per image) → visual trace (up to 5 points) → 256-bin action tokens, post-training chunk 8

Training

Pretraining accelerators

256 H100 (pre) + 128 H100 (mid)

Pretraining duration

Not disclosed (stated only as GPU-hours)

Pretraining steps & batch

Pre 100k steps·batch 512; mid 50k steps·batch 128 (paper)/256 (README)

GPU/TPU-hours

9,728 (Stated (approx.) · H100-hours (Sec 4.1; Sec 2.2 says 9,216))

Fine-tuning

LoRA r=32, LIBERO command 64 GPUs (8 nodes×8), global batch 128, 40k–80k steps; time not disclosed

Data type

Real-robot teleoperation + simulation·synthetic (auxiliary depth·trajectory annotations) + web vision-language

Data scale

Pre 26.3M samples (robot 10.5M, depth 1.5M, trajectory 1.5M, trajectory-conditioned 10.5M, web 2M); mid MolmoAct Dataset 10,689 trajectories·93 tasks

Main datasets

RT-1(Fractal), BridgeData V2, BC-Z, MolmoAct Dataset

Tasks & evaluation

SimplerEnv Google Robot visual matching zero-shot 70.5% (fine-tuned 71.6%, variant aggregation 72.1%); LIBERO 86.6%; real Franka single-arm +10%·bimanual +22.7% (vs π0-FAST)

Robot

Franka single arm · bimanual, Google robot (SimplerEnv), LIBERO simulation

Inference

Inference device & speed

Not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

14 GB

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Not used · Depth·trajectory reasoning supervised learning

Base control

Fixed · No base control (Franka arm·sim)

Notes

Key features

An 'action reasoning model' that outputs actions via depth perception and visual trajectories. Released weights·data·training code in full.

Notes

The 2,304 H100-hours of mid-training are excluded from the GPU-hours cell (12,032 combined). Pretraining GPU-hours differ within the paper: 9,728 (Sec 4.1) vs 9,216 (Sec 2.2). In-house data was collected by 5 full-time workers over 2 months.

Verification

Confirmed