VLA Gallery / All models / Navigation

OmniVLA

UC Berkeley·Toyota Motor North America·Princeton University · 2025-09

VLANavigation7.5Bregression

An omni-modal navigation VLA taking language·goal images·2D goal positions alone or combined, LoRA-trained from OpenVLA 7B on 9,500 h of data from 10 platforms.

Paper

OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot NavigationarXiv 2509.19480

Overview

Category

Navigation

Type

VLA

Released

2025-09

Organization

UC Berkeley·Toyota Motor North America·Princeton University

Open weights

Yes

Parameters

7.5B (based on OpenVLA; OmniVLA-edge 50M)

Architecture

Backbone LLM

Llama 2-7B · OpenVLA config

Vision encoder

DINOv2 + SigLIP

Action head

Linear action head (regresses 8 waypoints, OFT-style) (regression)

Chunk

8 waypoints (3 Hz · 2.4 s)

Control rate

Actions at 3 Hz intervals (execution rate not disclosed)

Architecture (original summary)

OpenVLA 7B (Llama 2-7B + DINOv2·SigLIP, 7.5B total) takes current·goal images, 2D goal position, language input (goal combinations masked via modality dropout) → linear action head on LLM output (OpenVLA-OFT style) regresses 8 waypoints (3 Hz, 2.4 s) → linear·angular velocity commands; lightweight OmniVLA-edge 50M (ViNT-based, EfficientNet-B0·CLIP·FiLM)

Training

Pretraining accelerators

8 H100 80GB (README states 'across 8 nodes')

Pretraining duration

Not disclosed

Pretraining steps & batch

Batch 7 per GPU × gradient accumulation 4 = effective batch 224; steps not disclosed

GPU/TPU-hours

Not disclosed

Fine-tuning

Version fine-tuned on CAST data released (GPU and time not disclosed)

Data type

Real-robot teleoperation / real-robot autonomous driving logs / web video / human video

Data scale

9,500 hours · 10 platforms (hours used: GNM 62.0h, LeLaN 128.7h, FrodoBots-2K 700h, BDD-V 8,680h)

Main datasets

GNM mix; LeLaN mix; FrodoBots-2K (MBRA relabeled); BDD-V (MBRA relabeled); CAST (fine-tuning)

Tasks & evaluation

Real-robot success rate: language 0.73 (OmniVLA-edge 0.60, LeLaN 0.43, CounterfactualVLA 0.33, MiniVLA 0.23, SmolVLA 0.10); 2D position 0.95 (MBRA 0.86, NoMaD 0.33); goal image 1.00 (MBRA 1.00, NoMaD 0.63, ViNT 0.50)

Robot

FrodoBots EarthRover Zero wheeled, VizBot wheeled, Unitree Go1 quadruped

Inference

Inference device & speed

Field deployment: a local RTX 4090 PC receives front video and position over the internet and sends velocity commands; latency and control rate not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

15 GB

Where it runs

Remote local PC RTX 4090 (over the internet)

Reinforcement learning & base

Reinforcement learning

Not used · Imitation learning only (including MBRA·NoMaD synthetic action labels)

Base control

Wheeled·Quadruped (Waypoints → tracking controller) · VLA outputs 8 waypoints → selected waypoint converted to linear·angular velocity commands and sent to the robot

Notes

Key features

An omni-modal navigation VLA taking language·goal images·2D goal positions alone or combined, LoRA-trained from OpenVLA 7B on 9,500 h of data from 10 platforms. On real robots, language-goal success 0.73 (LeLaN 0.43)·2D position 0.95·goal image 1.00, equal to or above modality-specific policies.

Notes

LoRA limits trained parameters to about 5%. FrodoBots·BDD-V use MBRA, LeLaN uses NoMaD synthetic action labels. HF page auto-label '8B params' (paper 7.5B), BF16 safetensors about 15.1GB. Training time·inference latency not disclosed. ICRA 2026. Based on: OpenVLA: An Open-Source Vision-Language-Action Model (arXiv 2406.09246) — training starts from the OpenVLA 7B checkpoint, based on the OpenVLA-OFT codebase (README)

Verification

Confirmed