LightNav-0

Light Origins · 2026-08

VLANavigation4.85Bdiscrete tokens

Adds only pointing tokens and RVQ action tokens to a pretrained VLM, with no new head, unifying instruction following·object search·visual tracking in one token interface, trained in the order ER mid-training→SFT→GRPO RL.

View architecture diagram in the gallery Paper ↗

Paper

LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied NavigationarXiv 2608.30935

Overview

Category: Navigation
Type: VLA
Released: 2026-08
Organization: Light Origins
Open weights: Yes
Parameters: 4.85B (HF safetensors) / 4B (based on Qwen3-VL-4B-Instruct; LightNav-ER 4B in README table, no sentence giving the total)

Architecture

Backbone LLM: Qwen3-VL-4B-Instruct (36-layer LLM)
Vision encoder: Qwen3-VL native-resolution ViT
Action head: 2 pointing tokens + 3 RVQ action tokens → 10 SE(2) waypoints (discrete tokens)
Chunk: 10 waypoints
Control rate: About 4–5 Hz (README closed loop)
Architecture (original summary): Qwen3-VL-4B-Instruct (native-resolution ViT + 36-layer LLM), vocabulary expanded without task-specific heads; per step 2 dual-channel pointing tokens (affordance·object) → 3 RVQ action tokens (codebook K=256×3) → decoded into 10-step SE(2) waypoints; recency-based history compression; robot-side MPC tracks the first waypoint, closed loop about 4–5 Hz

Training

Pretraining accelerators: H100 (count not disclosed)
Pretraining duration: Not disclosed
Pretraining steps & batch: ER mid-training batch 128 · max 10,240 tokens · lr 1e-5, SFT batch 320 · max 8,192 tokens (about 8.6 samples packed per sequence) · lr 1.5e-5; steps not disclosed
GPU/TPU-hours: 1,120 (Stated · H100-hours (sum of ER mid-training "approximately 170 H100 GPU-hours" + SFT "approximately 950 H100 GPU-hours"))
Fine-tuning: GRPO online RL: single node of 8 H100, time not disclosed; 256 episodes per update (G=8 × B=32), lr 1e-6
Data type: Simulation·synthetic / web vision-language
Data scale: Navigation 2K+ scenes·4K+ hours; ER mid-training 36 sources (pointing 35%·single-image VQA 25%·general reasoning 20%·video reasoning 20%); SFT uses 16 nav + 33 ER/VQA sources (nav supervision about 77.6%) + DAgger-collected data; blog: Stage II 742K instructions (474K unique)
Main datasets: R2R; RxR; ScaleVLN; HM3D; MP3D; VLNVerse; Habitat-GS; InteriorGS; HM3D-OVON; EVT-Bench; ER: RefSpatial, RoboPoint, VSI-590K, SQA3D, etc.
Tasks & evaluation: R2R-CE val-unseen SR 68.5%·SPL 62.8, RxR-CE SR 73.6%·SPL 64.5; ObjectNav MP3D 53.3%·HM3D v1 74.5%·v2 77.2%, HM3D-OVON unseen 47.0%; EVT-Bench STT SR 91.7%·DT SR 82.6%; INSIGHT-Bench SR 43.7%; LightNav-ER average 67.4 over 8 ER benchmarks; zero-shot on 4 real robots
Robot: Unitree Go2 quadruped, LimX TRON 1, LightBot-0, aerial robot (model not stated); paper describes only 4 types: humanoid·quadruped·aerial·wheeled

Inference

Inference device & speed: About 4 ms per generated token on RTX 4090 (vLLM, paper); README: about 60–150 ms per step on a single GPU, Jetson AGX Thor onboard bf16 268 ms (3.7 Hz)·fp8 (LLM only) 178 ms (5.6 Hz)·closed loop 4.5–4.8 Hz; robot-side MPC 10 Hz
Inference memory: Not disclosed
Weights bf16 (calc.): 9.7 GB
Where it runs: Remote GPU server or onboard Jetson AGX Thor

Reinforcement learning & base

Reinforcement learning: Used (VLA RL post-training) · GRPO online RL after SFT (task-level terminal reward, group-normalized advantage)
RL evidence: https://arxiv.org/html/2608.30935v2
Base control: Wheeled·Quadruped·Humanoid legged·Aerial (Waypoints → tracking controller) · Model outputs 10-step SE(2) waypoints → robot-side MPC tracks the first waypoint as linear·angular velocity commands (Go2·TRON1 adapters); also demonstrated on aerial robots

Notes

Key features: Adds only pointing tokens and RVQ action tokens to a pretrained VLM, with no new head, unifying instruction following·object search·visual tracking in one token interface, trained in the order ER mid-training→SFT→GRPO RL. With monocular RGB only, it achieved the best monocular success rates on 10 public sim settings (R2R-CE SR 68.5%, EVT-Bench STT SR 91.7%) and transferred zero-shot to humanoid·quadruped·aerial·wheeled robots.
Notes: GPU-hours sum two stated values (RL stage time not disclosed). Data hours (h) is the '4K+ hours' lower bound. Weights HF LightOriginsHQ/LightNav-0, code Apache 2.0, INSIGHT-Bench released separately. The blog (2026-09-01) describes converting 2,000+ internet real-world scenes into reusable sim worlds. Type classification of TRON 1 · LightBot-0 not stated in the original.
Verification: Corrections applied