VLA Gallery / All models / Mobile manipulation

MoManipVLA

Beijing University of Posts and Telecommunications·Nanyang Technological University·Tsinghua University · 2025-03

VLAMobile manipulation7Bdiscrete tokens

Transfers a VLA to mobile manipulation by using bi-level optimization to find base waypoints that make the end-effector waypoints from a fixed-base pretrained VLA feasible.

Paper

MoManipVLA: Transferring Vision-language-action Models for General Mobile ManipulationarXiv 2503.13446

Overview

Category

Mobile manipulation

Type

VLA

Released

2025-03

Organization

Beijing University of Posts and Telecommunications·Nanyang Technological University·Tsinghua University

Open weights

No

Parameters

7B (OpenVLA-7B)

Architecture

Backbone LLM

Llama 2 7B · OpenVLA-7B config

Vision encoder

DINOv2 + SigLIP (built into OpenVLA)

Action head

OpenVLA action tokens → end-effector waypoints; base waypoints·end-effector trajectory via bi-level optimization (discrete tokens)

Chunk

Not disclosed

Control rate

Not disclosed (about 0.58–0.59 s per real-world step)

Architecture (original summary)

OpenVLA-7B (LoRA fine-tuned) predicts end-effector waypoints from RGB·arm proprioception·language instructions → bi-level optimization: upper level chooses base waypoints, lower level chooses end-effector trajectory (reachability·smoothness·collision costs λ 10.0·1.0·0.6, Dual Annealing + SLSQP) → arm joints via Pinocchio IK; chunk·frequency not disclosed

Training

Pretraining accelerators

N/A (uses the public OpenVLA-7B)

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

4 RTX 3090, LoRA 10K epochs (200 sim demo episodes), time·batch not disclosed; real environment fine-tuned on 50 expert episodes

Data type

Simulation·synthetic / real-robot teleoperation†

Data scale

200 sim pick-and-place demonstration episodes (collected with the OVMM heuristic baseline); 50 real-world expert episodes

Main datasets

OVMM sim expert trajectories (self-collected)

Tasks & evaluation

OVMM (60 sim scenes) success rate 15.8%·partial success 49.4% (KUZHUM 11.6%·38.2%, UniTeam 9.2%, OVMM RL 1.2%); 3 real-world tasks, 10 trials each: Stack Block 30%, Open Drawer 10%, Put in Bowl 40%

Robot

Sim Hello Robot Stretch (OVMM); real hexman echo plus wheeled base + RM65 arm, base pose tracked with RealSense T265

Inference

Inference device & speed

Real-world per-step latency 580–592 ms (Table 4); inference hardware not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

14 GB

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Not used · Imitation learning (OpenVLA LoRA fine-tuning) + non-learned optimization

Base control

Wheeled (Motion set by planning/optimization) · Base waypoints computed by bilevel optimization drive translation·rotation; a fixed-base VLA outputs end-effector waypoints and the arm tracks them via IK

Notes

Key features

Transfers a VLA to mobile manipulation by using bi-level optimization to find base waypoints that make the end-effector waypoints from a fixed-base pretrained VLA feasible. On OVMM it reached 15.8% success, 4.2%p above the previous best (11.6%), and real-world deployment used only 50 expert episodes.

Notes

The base is driven by optimization, not a learned policy (the learned part is OpenVLA LoRA adaptation). The abstract's '50 training cost' means 50 expert episodes per the main text; the project page says '50% training cost'. Real-data collection method Not stated. Table 2 latency 693.1 has no unit stated. Code 'Coming Soon' (checked 2026-10), weights not released. CVPR 2025. Based on: OpenVLA: An Open-Source Vision-Language-Action Model (arXiv 2406.09246) — uses pretrained OpenVLA-7B as the end-effector waypoint generator

Verification

Confirmed