VLA Gallery / All models / π family

Hi Robot

Physical Intelligence, Stanford, UC Berkeley · 2025-02

VLAπ familyHigh-level PaliGemma-3B + low-level π0 3.3Bflow matching

A hierarchical system in which a high-level VLM turns user prompts and mid-task interventions into 1–3 s skill-level language commands executed by a low-level π0.

Paper

Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action ModelsarXiv 2502.19417

Overview

Category

π family

Type

VLA

Released

2025-02

Organization

Physical Intelligence, Stanford, UC Berkeley

Open weights

No

Parameters

High-level PaliGemma-3B + low-level π0 3.3B; system total not disclosed

Architecture

Backbone LLM

Gemma 2B × 2 (high-level · low-level, PaliGemma config)

Vision encoder

SigLIP 400M

Action head

π0 flow matching expert (flow matching)

Chunk

Not disclosed

Control rate

Not disclosed

Architecture (original summary)

High level: PaliGemma-3B VLM (outputs language commands, re-run every 1 s or on user intervention); low level: π0 (PaliGemma-3B + flow matching action expert)

Training

Pretraining accelerators

N/A (reuses PaliGemma-3B and π0 initialization)

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

High-level policy 8×H100 about 2 h, batch 512, lr 1×10⁻⁵ (1,000-step warmup); low-level policy time not disclosed

Data type

Real-robot teleoperation (segmented into 1–3 s skills) + VLM-generated synthetic language data (prompts·interventions)

Data scale

Not disclosed (hours·episodes·number of synthetic examples all)

Main datasets

In-house teleoperation demonstrations (name not disclosed)

Tasks & evaluation

Table bussing, sandwich making (up to 6 ingredients), grocery shopping; 20 trials per task·method; instruction accuracy on average 40%+ higher than GPT-4o high-level policy

Robot

UR5e (6-DoF, parallel gripper), bimanual ARX (6-DoF×2), mobile ARX (Mobile ALOHA)

Inference

Inference device & speed

1~2 RTX 4090; low-level onboard 73 ms·remote (WiFi) 86 ms; high-level RTX 4090 47 ms (prefill) + 13.2 ms (decode), H100 17.3 + 5.7 ms

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Local PC

Reinforcement learning & base

Reinforcement learning

Not used · Supervised learning (including synthetic interaction data)

Base control

Wheeled (Direct base velocity/pose output) · Low-level π0 policy output includes 2 dims for the mobile ARX base (grocery shopping); high-level VLM issues language commands only

Notes

Key features

A hierarchical system in which a high-level VLM turns user prompts and mid-task interventions into 1–3 s skill-level language commands executed by a low-level π0. The high-level policy was trained on user interaction data synthesized by a VLM.

Notes

Training compute and inference details appear only in arXiv v2 (2025-07-15). Name of the VLM used for synthetic data generation not disclosed. Parameters (B) is null since the system total is not disclosed.

Verification

Corrections applied