VLA Gallery / All models / Navigation

GNM

UC Berkeley·Toyota Motor North America · 2022-10

VANavigation8.7Mregression

Unifies heterogeneous driving data from 6 robots with a normalized action space and temporal context to train a single image-goal-conditioned navigation policy.

Paper

GNM: A General Navigation Model to Drive Any RobotarXiv 2210.03370

Overview

Category

Navigation

Type

VA

Released

2022-10

Organization

UC Berkeley·Toyota Motor North America

Open weights

Yes

Parameters

8.7M (trainable parameters; not stated in the GNM paper, given in the appendix of the same team's ViNT paper)

Architecture

Backbone LLM

None

Vision encoder

MobileNetV2 ×2 (observation context / observation+goal)

Action head

5-waypoint (x, y, yaw) regression + temporal distance head (regression)

Chunk

5 waypoints

Control rate

4 Hz (deployment setting; not stated in paper)

Architecture (original summary)

Two MobileNetV2 encoders (stacked past 5-frame observations / observation+goal image) → concatenated embeddings·3 FC layers → 2 heads: regression of 5 normalized relative waypoints (x, y, yaw change) + temporal distance to goal; 85×64 RGB; tracked by each robot's low-level controller

Training

Pretraining accelerators

Multi-GPU (type·count Not disclosed)

Pretraining duration

Not disclosed

Pretraining steps & batch

Batch 400–1200 (paper); public gnm_large config 30 epochs · batch 400

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A (deployed without per-robot fine-tuning)

Data type

Real-robot teleoperation / real-robot autonomous driving logs

Data scale

Over 60 hours (abstract · body)/70 hours (Table I); 6 robot platforms; 8 datasets (body)/9 (Table I)

Main datasets

GoStanford; RECON; CoryHall; Berkeley; SCAND; Seattle; TartanDrive; NeBula

Tasks & evaluation

Real-robot image-goal navigation success rate (GNM-Mid): LoCoBot 0.96·Tello 0.99·Vizbot 0.93·Jackal 0.94 (single dataset GS 0.26·0.21·0.51·0.31, RECON 0.62·0.79·0.26·0.68); long-range navigation with a topological graph

Robot

Evaluation: LoCoBot, Vizbot, Clearpath Jackal wheeled; DJI Tello quadrotor (restricted to a horizontal plane 1 m above ground, no flight data in training); training data: 6 robot platforms

Inference

Inference device & speed

Not stated in paper; official deployment config observes at 4 Hz (robot.yaml); waypoints tracked by each robot's low-level controller

Inference memory

Not disclosed

Weights bf16 (calc.)

0 GB

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Not used · Supervised imitation learning only (action and distance heads trained jointly with supervision)

Base control

Wheeled·Aerial (Waypoints → tracking controller) · Policy outputs 5 normalized relative waypoints → per-robot low-level controllers (PID, MPPI, etc.) track them; Tello drone restricted to a horizontal plane 1 m above ground

Notes

Key features

Unifies heterogeneous driving data from 6 robots with a normalized action space and temporal context to train a single image-goal-conditioned navigation policy. Drove 4 robots, including a Tello quadrotor unseen in training, without per-robot fine-tuning, and in some cases performed up to 5× better than single-dataset policies.

Notes

Data hours mismatch: 60h in main text vs 70h in Table I; dataset count also differs (8/9). Parameter count is not in the GNM paper but given as 8.7M in the appendix of the same group's ViNT paper. Small/Mid/Large differ in the number of datasets used (2/4/6), not model size. Checkpoints released on Google Drive; code merged into visualnav-transformer (README gives a LoCoBot+Jetson Orin Nano deployment example). ICRA 2023.

Verification

Confirmed