VLA Gallery / All models / Mobile manipulation

PanoVLA

Beihang University·Beijing Institute for General Artificial Intelligence(BIGAI)·Shandong University·Johns Hopkins University·Delta Intelligence · 2026-08

VLAMobile manipulationNot disclosedflow matching

A Mixture-of-Transformers inserting a panorama expert into π0.5 fuses global spatial context from a 360° panorama with local camera views, and collects wheeled bimanual robot data via single VR whole-body teleoperation.

Paper

Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body TeleoperationarXiv 2608.02257

Overview

Category

Mobile manipulation

Type

VLA

Released

2026-08

Organization

Beihang University·Beijing Institute for General Artificial Intelligence(BIGAI)·Shandong University·Johns Hopkins University·Delta Intelligence

Open weights

Unverified

Parameters

Not disclosed (panorama expert 100M; π0.5 backbone Gemma-2B)

Architecture

Backbone LLM

Gemma-2B · π0.5 VLM config

Vision encoder

SigLIP (local view) + MTPano (panorama, frozen)

Action head

flow matching action expert (inherited from π0.5) (flow matching)

Chunk

32 steps

Control rate

Not disclosed (data at 25 Hz)

Architecture (original summary)

π0.5-based 3-expert Mixture-of-Transformers: VLM expert (SigLIP local view + Gemma-2B) + panorama expert 100M (dual fisheye → equirectangular panorama, frozen MTPano features) + flow matching action expert; 17-dim relative actions (bimanual joints·grippers 14 + base planar velocity·yaw rate 3) 32-step chunk; data 25 Hz

Training

Pretraining accelerators

N/A (fine-tuned from π0.5)

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

4 × A100, 30K steps · batch 64 per GPU (bf16, lr fixed at 5e-5 after 5K-step warmup), LM · MTPano frozen, training time not disclosed

Data type

Real-robot teleoperation

Data scale

4 tasks × 200 = 800 trajectories, about 5.5 hours · about 500k frames (25 Hz, 20–30 s per trajectory)

Main datasets

In-house VR whole-body teleoperation dataset (Move Pen, Move Block, Open Curtain, Wipe Table)

Tasks & evaluation

Real 4 tasks, 15 trials each: Move Pen SR 86.7%, Move Block 93.3%, Open Curtain 66.7%, Wipe Table 46.7%; average SCR 91.3%·SR 73.4% (π0.5 58.6%·30.0%, π0.5 w/ Pano 80.8%·56.7%)

Robot

Wheeled bimanual: RANGER MINI 3 base + two Agilex PIPER 6-DoF arms·parallel grippers; top 360° panorama camera·front RGB·2 wrist cameras

Inference

Inference device & speed

Closed-loop execution on 1 RTX 4090 during real-robot evaluation; latency·control frequency not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Single RTX 4090

Reinforcement learning & base

Reinforcement learning

Not used · Imitation learning (flow matching) only

Base control

Wheeled (Direct base velocity/pose output) · Policy directly outputs base planar velocity and yaw rate (3-DoF) together with both arms' joints as 32-step chunks

Notes

Key features

A Mixture-of-Transformers inserting a panorama expert into π0.5 fuses global spatial context from a 360° panorama with local camera views, and collects wheeled bimanual robot data via single VR whole-body teleoperation. On 4 real tasks it reached an average stage completion of 91.3%·end-to-end success of 73.4%, well ahead of π0.5 (58.6%·30.0%).

Notes

Teleoperation: VR headset, two hand controllers, 3 waist/ankle trackers → SMPL reconstruction → GMR retargeting (base is a 3-DoF planar virtual floating base). π0.5 checkpoint name, total parameters, denoising steps, and training time not disclosed. The paper does not mention releasing code, data, or weights. Based on: π0.5: a Vision-Language-Action Model with Open-World Generalization (arXiv 2504.16054) — built by inserting a panoramic expert into the π0.5 dual-expert backbone

Verification

Confirmed