VLA Gallery / All models / Mobile manipulation

SG-VLA

Lambda, Inc.·UC San Diego · 2026-03

VLAMobile manipulation1.3Bdiscrete tokens

Strengthens spatial grounding by co-training, in stages with the VLA, auxiliary decoders that reconstruct robot global pose·joint configuration·grasp success·target object pose·segmentation masks.

Paper

SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile ManipulationarXiv 2603.22760

Overview

Category

Mobile manipulation

Type

VLA

Released

2026-03

Organization

Lambda, Inc.·UC San Diego

Open weights

Unverified

Parameters

1.3B (Prismatic VLM based on Qwen2.5-0.5B; optional flow expert 100M separate)

Architecture

Backbone LLM

Qwen2.5-0.5B · Prismatic VLM config

Vision encoder

DINOv2 + SigLIP (Prismatic; RGB · depth multi-view)

Action head

Discrete action tokens + 5 auxiliary decoders; optional flow matching expert 100M (10 steps) (discrete tokens)

Chunk

Not disclosed (token path) / flow head runs 2 of 8 steps

Control rate

Not disclosed

Architecture (original summary)

Prismatic VLM (DINOv2 + SigLIP fused encoder, Qwen2.5-0.5B LLM); input head·hand multi-view RGB·depth·past 4-step history; predicts 13-dim actions (base pose ΔX 3·torso height 1·arm joint Δq 7·gripper 2) as discrete action tokens; optional 100M flow matching expert (execute 2 of 8-step chunk, 10 denoising steps); co-trained with 5 auxiliary decoders

Training

Pretraining accelerators

N/A (trained directly on task data from a pretrained Prismatic VLM)

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

8 × A100, 3 stages (decoder adaptation → joint refinement → action head only with VLM frozen), 10 epochs on 6 subtasks (5 epochs pick · place for the split-decoder model) · global batch 512 · Adam fixed lr 2e-5, time not disclosed

Data type

Simulation·synthetic

Data scale

44K episodes · 1.4M transitions

Main datasets

ManiSkill-HAB data generation pipeline (TidyHouse·PrepareGroceries·SetTable)

Tasks & evaluation

ManiSkill-HAB (TidyHouse·PrepareGroceries·SetTable → Pick·Place·Open·Close) 30 trials per task: average 0.73 (no auxiliary tasks 0.60, naive co-training 0.51, flow head 0.69; OpenVLA 0.04, multi-view+depth OpenVLA 0.32); no real-robot experiments

Robot

ManiSkill-HAB sim mobile manipulator (Fetch per MS-HAB; SG-VLA paper does not name the robot), 13-dim control of base·torso·7-joint arm·gripper

Inference

Inference device & speed

Not disclosed

Inference memory

Not disclosed

Weights bf16 (calc.)

2.6 GB

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Partial (RL-generated data) · Training data generated by the ManiSkill-HAB generation pipeline (RL policy demonstrations + rule-based filtering); the policy itself uses imitation learning

RL evidence

https://arxiv.org/abs/2412.13211

Base control

Wheeled (Direct base velocity/pose output) · Policy directly outputs base pose change (3-dim ΔX) together with torso, arm, and gripper as 13-dim actions (sim)

Notes

Key features

Strengthens spatial grounding by co-training, in stages with the VLA, auxiliary decoders that reconstruct robot global pose·joint configuration·grasp success·target object pose·segmentation masks. Average success on ManiSkill-HAB household tasks was 0.73, 22% higher than a multi-view·depth model without auxiliary tasks (0.60) and far ahead of OpenVLA (0.04).

Notes

Sim only (the conclusion names real-world application as future work). The paper does not mention code·weight release and no repository was found by search. History length is given inconsistently as '4 timesteps' and '4 actions'. Training time·inference speed not disclosed. Compute supported by Lambda. Based on: Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models (arXiv 2402.07865) — uses the pretrained Prismatic VLM as backbone

Verification

Confirmed