VLA Gallery / All models / Reference baselines (pre-2024)

RT-1-X

Open X-Embodiment Collaboration (21 institutions) · 2023-10

VAReference baselines (pre-2024)35Mdiscrete tokens

A cross-embodiment model training the RT-1 architecture on mixed OXE data from 9 embodiments.

Paper

Open X-Embodiment: Robotic Learning Datasets and RT-X ModelsarXiv 2310.08864

Overview

Category

Reference baselines (pre-2024)

Type

VA

Released

2023-10

Organization

Open X-Embodiment Collaboration (21 institutions)

Open weights

Yes

Parameters

35M

Architecture

Backbone LLM

None

Vision encoder

EfficientNet-B3 + TokenLearner

Action head

256-bin discrete tokens (discrete tokens)

Chunk

None (1 step)

Control rate

3–10 Hz (per robot)

Architecture (original summary)

RT-1 architecture (FiLM EfficientNet-B3 + TokenLearner + Transformer); 8-dim (1 terminate·7 end-effector motion) 256-bin tokens

Training

Pretraining accelerators

Not disclosed

Pretraining duration

Not disclosed

Pretraining steps & batch

Not disclosed

GPU/TPU-hours

Not disclosed

Fine-tuning

Not disclosed

Data type

Real-robot teleoperation etc. (collection method varies by contributed dataset)

Data scale

OXE 1M+ real-robot trajectories, 22 embodiments, 60 datasets (34 labs), 527 skills (160,266 tasks); training mix uses 9 embodiments

Main datasets

OXE subset: RT-1, QT-Opt, Bridge, Task Agnostic Robot Play, Jaco Play, Cable Routing, RoboTurk, NYU VINN, Austin VIOLA, Berkeley Autolab UR5, TOTO, Language Table

Tasks & evaluation

3,600 evaluations on 6 robot types; average success in small-data domains 50% higher than the original methods·RT-1

Robot

Training mixture of 9 manipulators (out of 22 embodiments in all of OXE); 6 evaluation robots

Inference

Inference device & speed

3–10 Hz depending on robot; RT-1-X runs locally

Inference memory

Not disclosed

Weights bf16 (calc.)

0.1 GB

Where it runs

Local PC

Reinforcement learning & base

Reinforcement learning

Not used · Imitation learning. The training mix includes the QT-Opt dataset (collected by an RL policy)

Base control

Fixed · No base control (8-dim actions: terminate 1 + EE movement 7; Google mobile robot is arm only too)

Notes

Key features

A cross-embodiment model training the RT-1 architecture on mixed OXE data from 9 embodiments. In small-data domains, its average success rate was 50% higher than the original methods and RT-1.

Notes

TF·JAX checkpoints released (open_x_embodiment README). Institution listing corrected from 'Google DeepMind + 21 institutions' to the full 21-institution collaboration.

Verification

Corrections applied