VLA Gallery / All models / Navigation

FutureNav

Tsinghua University·Pengcheng Laboratory·HKUST (Guangzhou)·Xiaomi EV·National University of Singapore · 2026-06

VLANavigation4B · 8Bdiscrete tokens

Brings world-action modeling to VLN, jointly learning action prediction, inverse·forward dynamics, and future spatial state generation on VLM hidden states, while using only the policy head at inference to keep speed.

Paper

FutureNav: Unified World-Action Modeling for Vision-and-Language NavigationarXiv 2606.30367

Overview

Category

Navigation

Type

VLA

Released

2026-06

Organization

Tsinghua University·Pengcheng Laboratory·HKUST (Guangzhou)·Xiaomi EV·National University of Singapore

Open weights

Partial

Parameters

4B · 8B (by Qwen3-VL-4B · 8B backbone; HF FutureNav-4B-Base totals 5.48B incl. VGGT · heads, 8B total not disclosed)

Architecture

Backbone LLM

Qwen3-VL-4B / 8B

Vision encoder

Qwen3-VL vision encoder (frozen) + VGGT spatial encoder (frozen)

Action head

Discrete text actions (STOP·MOVE_FORWARD·TURN_LEFT·TURN_RIGHT) (discrete tokens)

Chunk

None (1 step)

Control rate

Not disclosed

Architecture (original summary)

Qwen3-VL-4B/8B (vision tower frozen) + frozen VGGT spatial encoder (mid-layer tokens, 2048 dims per patch → RMSNorm·2-layer MLP merger, added to visual tokens with α=0.2) → 4 heads on LLM hidden states (autoregressive action policy, inverse dynamics classification, forward dynamics MLP, future spatial feature generation); inference uses only the policy head, one discrete text action (STOP·MOVE_FORWARD·TURN_LEFT·TURN_RIGHT) per step

Training

Pretraining accelerators

Not disclosed

Pretraining duration

Not disclosed

Pretraining steps & batch

Not disclosed

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A (single navigation training stage from Qwen3-VL; accelerators and time not disclosed)

Data type

Simulation·synthetic

Data scale

VLN-CE expert trajectories about 2.43M samples (R2R-CE 631K + RxR-CE 1.80M, '0K' setting); full setting adds 10,457K (4B)/10,435K (8B) external data from ScaleVLN·DAgger rollouts

Main datasets

R2R-CE; RxR-CE; ScaleVLN; DAgger rollouts

Tasks & evaluation

R2R-CE val-unseen SR 65.4%·SPL 61.3·NE 4.24 (4B full data; 8B 64.3%·60.3), RxR-CE SR 60.9%·SPL 52.8·nDTW 68.0 (8B SR 63.9%·SPL 54.8); 0K setting without external data 4B R2R SR 55.1%; real-robot Go2 zero-shot (no quantitative numbers reported)

Robot

Go2 quadruped (teleoperated, D435i RGB camera); sim VLN-CE agent

Inference

Inference device & speed

Real robot uses remote inference on an H20 GPU server (Go2 camera RGB streaming, one low-level action returned per step); latency·Hz·memory not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

16 GB

Where it runs

Remote server (H20)

Reinforcement learning & base

Reinforcement learning

Not used · Supervised learning (cross-entropy, MSE); DAgger rollouts used only as training data

Base control

Quadruped (Discrete motion actions) · Policy outputs one discrete action (stop, forward, turn left/right) per step → remote server relays it to Go2 for execution (low-level control not stated)

Notes

Key features

Brings world-action modeling to VLN, jointly learning action prediction, inverse·forward dynamics, and future spatial state generation on VLM hidden states, while using only the policy head at inference to keep speed. With monocular RGB·4B backbone it reaches R2R-CE SR 65.4%·RxR-CE SR 60.9%, outperforming prior VLN models such as 7B-class JanusVLN (R2R 60.5%).

Notes

Training hardware, time, steps, batch, and lr all not disclosed. HF llxs/FutureNav releases only FutureNav-4B-Base (11.7 GB); 8B, training scripts, and training data not released (README TODO). Real robots only in demos, no quantitative results. Composition of the external data column (10457K etc.) not explained in the text.

Verification

Corrections applied