TRI LBM 1.0

Toyota Research Institute · 2025-07

VLAIndustry modelsNot discloseddiffusion

A diffusion transformer policy conditioned on CLIP language without an LLM·VLM.

Paper

Backbone LLM: None (CLIP text encoder)
Vision encoder: CLIP ViT-B/16
Action head: AdaLN DiT 8 blocks (diffusion) (diffusion)
Chunk: 16 (8 executed)
Control rate: 10 Hz
Architecture (original summary): CLIP ViT-B/16 image CLS + CLIP text encoder → AdaLN-conditioned Diffusion Transformer 8 blocks (embedding 768); predicts 16 steps × 20-dim actions, recomputes after executing 8 steps

Pretraining accelerators: Not disclosed
Pretraining duration: Not disclosed
Pretraining steps & batch: 48k steps, global batch 2560, lr 3e-4 (constant)
GPU/TPU-hours: Not disclosed
Fine-tuning: GPU and time not disclosed; real-robot tasks 30k steps, simulation tasks 10k steps, global batch 320, lr 2e-5
Data type: Real-robot teleoperation + simulation teleoperation + UMI handheld demonstrations + public robot data (OXE)
Data scale: About 1,700 h (abstract; §III: ~1695 h): TRI-Ramen 545 h·532 tasks·64,262 demos (Real 468 h·362 tasks·46,063 / Sim 45 h·41 tasks·7,348 / UMI 32 h·129 tasks·10,851) + OXE-Ramen about 1,150 h
Main datasets: TRI-Ramen (Real·Sim·UMI, internal), OXE-Ramen (curated Open X-Embodiment)
Tasks & evaluation: 1,800 real-robot rollouts (50 per task·policy·condition), 47,000+ simulation rollouts (200 per condition); in simulation needs under 30% of the data vs scratch; on real robots, with 15% of the data it statistically beats single-task baselines trained on the full data
Robot: Bimanual Franka FR3 + parallel grippers (finray-type fingers), 9 stations, up to 6 cameras; simulation

Inference device & speed: Policy loop 10Hz (executes 8 of 16 predicted steps, then replans); hardware and latency not disclosed
Inference memory: Not disclosed
Weights bf16 (calc.): Not disclosed
Where it runs: Not disclosed

Reinforcement learning: Not used · Imitation learning (behavior cloning) only
Base control: Fixed · No base control (fixed bimanual Franka station)

Key features: A diffusion transformer policy conditioned on CLIP language without an LLM·VLM. Evaluated over 1,800 real-robot and 47,000+ simulation rollouts with Bayesian testing and CLD.
Notes: The number of simulation evaluation tasks differs by source: paper 16 seen + 8 unseen, project page 16 seen + 5 unseen (29 tasks including real robot). Instead of policy weights, the simulation benchmark lbm_eval (49 tasks) was released; whether policy weights are released could not be verified. Project page wording: '3-5× less data'. No GPU sentence in the paper·page.
Verification: Confirmed