VLA Gallery / All models / Industry models

AutoRT

Google DeepMind · 2024-01

기타Industry modelsN/AOther

A system that orchestrates a robot fleet with LLMs·VLMs to collect real-world data at scale; not a model.

Paper

AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic AgentsarXiv 2401.12963

Overview

Category

Industry models

Type

기타

Released

2024-01

Organization

Google DeepMind

Open weights

No

Parameters

N/A

Architecture

Backbone LLM

PaLI·FlexCap VLM + LLM (name not stated)

Vision encoder

—

Action head

Collected with RT-1·RT-2·scripted·teleoperation policies (Other)

Chunk

Not disclosed

Control rate

Not disclosed

Architecture (original summary)

System: VLMs (PaLI, FlexCap) describe the scene → LLM proposes·filters tasks → collected with RT-1/RT-2/scripted/teleoperation policies

Training

Pretraining accelerators

N/A

Pretraining duration

N/A

Pretraining steps & batch

N/A

GPU/TPU-hours

Not disclosed

Fine-tuning

N/A (RT-1 co-finetuning on collected data, compute not disclosed)

Data type

Autonomous real-robot collection (scripted policies·teleoperation·RT-2)

Data scale

77,000 trials, 6,650 unique instructions, 7 months; scripted 73,293 episodes (21% success), teleop 3,060 (82%), RT-2 936 (4.7%)

Main datasets

AutoRT-collected data (in-house)

Tasks & evaluation

With RT-1 co-finetuning, height-generalization picking 0%→12.5%, wiping 10%→30%

Robot

20+ mobile manipulators operated simultaneously (blog: 52 in total), 4 office buildings

Inference

Inference device & speed

N/A

Inference memory

Not disclosed

Weights bf16 (calc.)

Not disclosed

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Not used · Data collection system (collected with RT-1·RT-2 policies·scripts·teleoperation)

Base control

Fixed · Base movement uses a separate navigation stack (map-based goal sampling); collection policies (RT-2·scripts·teleoperation) handle manipulation only

Notes

Key features

A system that orchestrates a robot fleet with LLMs·VLMs to collect real-world data at scale; not a model. Filters tasks for safety using a 'robot constitution' based on modified Asimov principles.

Notes

A data collection system (not a model), so type is Other. Supervision ratio: 3~5 robots per person for mobile, up to 8 per person for fixed. The blog uses 'tasks' instead of 'instructions'.

Verification

Not verified