VLA Gallery / All models / Navigation

Uni-LaViRA

Nanjing University·Institute of Automation, Chinese Academy of Sciences·Beihang University·BMW (Nanjing) Information Technology Co., Ltd.·University of Rochester · 2026-05

기타NavigationNot disclosedOther

A training-free agent in which a pretrained MLLM picks a direction in language and points to a target pixel visually, which is translated into robot actions; adding TODO List Memory and Second Chance Backtrack, it handles 4 task families·4 robot types with one structure.

Paper

Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied NavigationarXiv 2605.27582

Overview

Category

Navigation

Type

기타

Released

2026-05

Organization

Nanjing University·Institute of Automation, Chinese Academy of Sciences·Beihang University·BMW (Nanjing) Information Technology Co., Ltd.·University of Rochester

Open weights

Partial

Parameters

Not disclosed (LA Gemini-3.1-Pro size undisclosed + VA Qwen3.5-27B; ground real robot Qwen3.5-27B-Q4·9B-Q4)

Architecture

Backbone LLM

Gemini-3.1-Pro (LA, sim · real UAV) + Qwen3.5-27B (VA); ground real robots use Qwen3.5-27B-Q4 / 9B-Q4 for both LA · VA

Vision encoder

MLLM built-in vision (Qwen3.5-27B VA grounds bbox·points)

Action head

Direction·tool-call text + pixel goal → 3D point → FMM/NavDP path (Other)

Chunk

Not disclosed

Control rate

30–60 s per LA call; G1 velocity commands at 1 Hz

Architecture (original summary)

Training-free three-stage translation: Language Action (Gemini-3.1-Pro; Qwen3.5-27B/9B-Q4 on real robots) picks a direction (forward, left, right, back) and tool calls (turn, backtrack, stop, etc.) → Vision Action (Qwen3.5-27B) marks a bbox/point on the view at the chosen moment → Robot Action back-projects it to a 3D point, then FMM (sim)/NavDP (real robot) path → per-robot controllers; discrete ground motion in sim (0.25 m forward, 30° turns), UAV (x,y,z) waypoints

Training

Pretraining accelerators

N/A

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A (no training, prompts only)

Data type

N/A (no training data)

Data scale

N/A (evaluation only: stratified sampling of 100 episodes × 3 seeds across 6 benchmarks)

Main datasets

Evaluation: R2R-CE; RxR-CE; HM3D-v2; HM3D-OVON; MP3D-EQA; OpenUAV (UM split)

Tasks & evaluation

Zero-shot R2R-CE SR 60.7%, SPL 47.7; RxR-CE SR 51.3%, SPL 34.0; HM3D-v2 ObjectNav SR 77.7%, SPL 46.1; HM3D-OVON SR 60.0%; MP3D-EQA accuracy 54.7%; OpenUAV SR 40.0% (100 episodes × 3 seeds per benchmark, 1,800 trajectories total); demonstrated on 4 real robots

Robot

Agilex Cobot Magic wheeled bimanual (2-DoF differential base), Unitree G1 humanoid, Unitree Go1 quadruped, custom-built quadrotor UAV

Inference

Inference device & speed

LA call 30–60 s (normal API)·80–150 s (contention), local planner·controller each under 10 ms; per LA call 22,305 input·1,618 output tokens; sim parallel rollouts on 8 RTX 4090; real robots: Cobot Magic onboard RTX 4090 (Qwen3.5-27B-Q4), G1·Go1 onboard Jetson Orin NX (Qwen3.5-9B-Q4)

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Sim: remote API; real robot: onboard RTX 4090 / Jetson Orin NX

Reinforcement learning & base

Reinforcement learning

Not used · No training (prompt-based agent); low-level NavDP uses an existing imitation learning model as is

Base control

Wheeled·Quadruped·Humanoid legged·Aerial (Waypoints → tracking controller) · Target pixels chosen by LA·VA back-projected to 3D points → NavDP trajectory tracked by each robot's default controller (Cobot Magic proportional velocity control, G1 (vx,vy,ωz) 1 Hz, Go1 velocity mode); UAV uses PX4 position control

Notes

Key features

A training-free agent in which a pretrained MLLM picks a direction in language and points to a target pixel visually, which is translated into robot actions; adding TODO List Memory and Second Chance Backtrack, it handles 4 task families·4 robot types with one structure. Without training on robot data, it reached R2R-CE SR 60.7%·HM3D-v2 ObjectNav SR 77.7%, matching or beating trained navigation foundation models.

Notes

No trained weights of its own (training-free framework): main sim results rely on the closed Gemini-3.1-Pro API, while VA Qwen3.5-27B, real-robot Qwen3.5 Q4, and low-level NavDP are open weights, hence marked 'Partial'. Code released (CC BY-NC-SA 4.0). The three real ground robots use NavDP instead of FMM for the Robot Action stage. Institution Beihang University per the project page author listing. Extends the prior work LaViRA (zero-shot VLN-CE). Based on: LaViRA: Language-Vision-Robot Actions Translation for Zero-Shot Vision Language Navigation in Continuous Environments (arXiv 2510.19655) — the official code README states 'This work builds on LaViRA', and the project page says 'Previously on LaViRA'

Verification

Corrections applied