NavFoM

Peking University·Galbot·USTC·BAAI·University of Adelaide·Zhejiang University·Differential Robotics · 2025-09

VLANavigation7Bregression

A navigation foundation model trained as one model on multiple embodiments (quadrupeds·drones·wheeled·vehicles etc.) and VLN·object search·tracking·autonomous driving tasks, introducing viewpoint·time instruction tokens and token-budget-based history sampling.

View architecture diagram in the gallery Paper ↗

Paper

Embodied Navigation Foundation ModelarXiv 2509.12129

Overview

Category: Navigation
Type: VLA
Released: 2025-09
Organization: Peking University·Galbot·USTC·BAAI·University of Adelaide·Zhejiang University·Differential Robotics
Open weights: No
Parameters: 7B (based on Qwen2-7B; overall figure not stated)

Architecture

Backbone LLM: Qwen2-7B
Vision encoder: DINOv2 + SigLIP
Action head: 3-layer MLP planning head: 8 waypoints (x, y, z, θ) + language head (QA) (regression)
Chunk: 8 waypoints
Control rate: Up to 0.5 s per trajectory (1,600-token budget)
Architecture (original summary): DINOv2 + SigLIP dual vision encoder (coarse · fine tokens, 576 patches per frame) + 2-layer MLP projector + Qwen2-7B; temporal-viewpoint indicator (TVI) tokens and budget-aware temporal sampling (BATS, budget 1,600–2,048); 3-layer MLP planning head regresses 8 waypoints (x, y, z, θ), language head for QA; 1–8 camera inputs

Training

Pretraining accelerators: 56 H100
Pretraining duration: About 72 h
Pretraining steps & batch: 1 epoch (batch and learning rate not disclosed)
GPU/TPU-hours: 4,032 (Stated · H100-hours ("a total of 4,032 GPU hours"))
Fine-tuning: N/A (no per-task fine-tuning)
Data type: Simulation·synthetic / web video / real-robot autonomous driving logs / web vision-language
Data scale: 12.7M instances total: navigation 8.02M (VLN 3.37M, ObjectNav 1.02M, active visual tracking 897K, autonomous driving 681K, web video navigation 2.03M) + image QA 3.15M + video QA 1.61M
Main datasets: VLN-CE R2R·RxR; OpenUAV (429K); HM3D ObjectNav (L3MVN successful episodes); EVT-Bench; nuScenes (27K)·OpenScene (654K) vehicle driving data; Sekai (about 182K YouTube videos, VLM·SLAM pseudo-labels)
Tasks & evaluation: R2R-CE SR 61.7%·SPL 55.3% (4 views)/56.2% (single view); RxR-CE SR 64.4% (4 views)/57.4% (single view); HM3D-OVON zero-shot val-unseen SR 45.2%; EVT-Bench single-target SR 88.4%·distracted-target 62.0%; NAVSIM PDMS 84.3 (8 views); OpenUAV seen SR 29.17%; 110 real-world trials (VLN 50·search 30·tracking 30)
Robot: Unitree Go2 quadruped (SG3S camera), Unitree G1 humanoid (RealSense D455), NCS-β aerial robot (RealSense D435), Galbot G1 wheeled (SG3S camera); trajectory tracking with each robot's off-the-shelf local planner

Inference

Inference device & speed: Runs on a remote server RTX 4090 (paper)/RTX 5090 (project page); the robot sends compressed observations; 8-waypoint trajectory generation up to 0.5 s with a 1,600-token budget
Inference memory: About 19.1 GB (ICLR camera-ready)
Weights bf16 (calc.): 14 GB
Where it runs: Remote server (RTX 4090/5090)

Reinforcement learning & base

Reinforcement learning: Not used · Imitation learning (SFT) only
Base control: Quadruped·Humanoid legged·Wheeled·Aerial (Waypoints → tracking controller) · Model outputs a waypoint trajectory → tracked by each robot's off-the-shelf local planner as velocity·joint commands (including drones)

Notes

Key features: A navigation foundation model trained as one model on multiple embodiments (quadrupeds·drones·wheeled·vehicles etc.) and VLN·object search·tracking·autonomous driving tasks, introducing viewpoint·time instruction tokens and token-budget-based history sampling. With 8.02M navigation samples and no per-task fine-tuning, it was best or competitive on R2R-CE (4-view SR 61.7%)·RxR-CE·HM3D-OVON·EVT-Bench·NAVSIM etc.
Notes: Inference GPU differs between the paper (RTX 4090) and the project page (RTX 5090); autonomous-driving data are real vehicle driving logs (nuScenes, OpenScene), with no real-robot training data collected; no public code or weights link (per project page); ICLR 2026.
Verification: Corrections applied