VLA Gallery / All models / Mobile manipulation

TidyBot++

TidyBot++ (diffusion policy)

Princeton University·Stanford University·Dexterity · 2024-11

VAMobile manipulationNot discloseddiffusion

Learns a per-task Diffusion Policy from phone teleoperation demonstrations on an omnidirectional (holonomic) powered-caster base, jointly outputting the base target pose and the arm end-effector target.

Paper

TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot LearningarXiv 2412.10447

Overview

Category

Mobile manipulation

Type

VA

Released

2024-11

Organization

Princeton University·Stanford University·Dexterity

Open weights

Partial

Parameters

Not disclosed

Architecture

Backbone LLM

None

Vision encoder

robomimic CNN encoder (ResNet-18†) · base·wrist cameras 84×84

Action head

DDIM diffusion head (100 training · 16 inference steps) (diffusion)

Chunk

executes 8 of 16 horizon steps

Control rate

10 Hz

Architecture (original summary)

Diffusion Policy (U-Net hybrid image, official code settings): robomimic CNN encoder (ResNet-18†) encodes 84×84 base · wrist cameras + base pose · arm state → 1D temporal U-Net, DDIM (100 training · 16 inference steps); 13-dim absolute target actions (base x · y · θ + arm end-effector position · 6D rotation + gripper), horizon 16 · 8 steps executed, 10 Hz

Training

Pretraining accelerators

N/A

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

Trained from scratch per task: GPU laptop (README: handles training·inference), 500 epochs (code default batch 64), training time not disclosed

Data type

Real-robot teleoperation

Data scale

6 tasks: open fridge 100 runs · other 5 tasks 50 runs each (350 total); 1–2 hours to collect 50 runs (incl. resets)

Main datasets

In-house phone teleoperation demonstrations (real apartments)

Tasks & evaluation

6 real apartment tasks, 10 trials each: open fridge 10/10, wipe countertop 9/10, load dishwasher 7/10, take out trash 10/10, load laundry 7/10, water plant 6/10; differential-drive-constrained policy wipe countertop 4/10

Robot

TidyBot++: powered-caster omnidirectional wheeled base (base $5–6k) + Kinova Gen3 7-DoF arm, Intel NUC mini PC

Inference

Inference device & speed

GPU laptop inference 115 ms (RTX 4080 Laptop, README), next chunk computed 200 ms ahead to hide latency; policy 10 Hz; real-time control on onboard Intel NUC

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

GPU laptop (RTX 4080 Laptop) inference + onboard Intel NUC control

Reinforcement learning & base

Reinforcement learning

Not used · Imitation learning only (per-task Diffusion Policy)

Base control

Wheeled (Direct base velocity/pose output) · Policy outputs base target pose (x, y, θ) together with arm targets → omnidirectional base low-level controller tracks it

Notes

Key features

Learns a per-task Diffusion Policy from phone teleoperation demonstrations on an omnidirectional (holonomic) powered-caster base, jointly outputting the base target pose and the arm end-effector target. It succeeded 6/10 to 10/10 on 6 tasks in real apartments, while a differential-drive-constrained policy under the same conditions reached only 4/10 on the mopping task (omnidirectional 9/10).

Notes

Paper describes the policy only as a 'diffusion policy'; architecture·frequency·latency follow the official GitHub code·README. Hardware design·teleoperation·training·control code open-sourced; weights released only for the sim sim-v1 policy checkpoint (real-task policies not released). Model size·training GPU model·time Not disclosed. Based on: Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (arXiv 2303.04137) — paper·README use Diffusion Policy (real-stanford/diffusion_policy code) as-is for policy training

Verification

Corrections applied