VLA Gallery / All models / Autoregressive & latent-action VLA

ECoT

ECoT (Embodied Chain-of-Thought)

UC Berkeley·Stanford·University of Warsaw · 2024-07

VLAAutoregressive & latent-action VLA7Bdiscrete tokens

Generates concrete reasoning such as plans·subtasks·object locations before actions, raising OpenVLA's absolute success rate by 28%.

Paper

Robotic Control via Embodied Chain-of-Thought ReasoningarXiv 2407.08693

Overview

Category

Autoregressive & latent-action VLA

Type

VLA

Released

2024-07

Organization

UC Berkeley·Stanford·University of Warsaw

Open weights

Yes

Parameters

7B (based on OpenVLA)

Architecture

Backbone LLM

Llama 2 7B · OpenVLA

Vision encoder

SigLIP + DINOv2

Action head

Reasoning text → 256-bin tokens (discrete tokens)

Chunk

None (1 step)

Control rate

Not disclosed

Architecture (original summary)

OpenVLA/Prismatic-7B; 256-bin action tokens after reasoning text (TASK·PLAN·SUBTASK·MOVE·GRIPPER·OBJECTS, including bounding boxes)

Training

Pretraining accelerators

Not disclosed (README example uses 8 GPUs)

Pretraining duration

Not disclosed

Pretraining steps & batch

Bridge reasoning-annotated data 80k steps (ecot-openvla-7b-bridge); batch not disclosed

GPU/TPU-hours

Not disclosed

Fine-tuning

OXE variant: after OXE action training, 20k additional steps on reasoning-mixed data; GPU·time·batch not disclosed

Data type

Real-robot teleoperation + simulation·synthetic (synthetic reasoning annotations)

Data scale

Bridge v2 2.5M+ transitions (number of annotated episodes not disclosed)

Main datasets

BridgeData V2, Embodied-CoT/embodied_features_bridge, OXE (variant)

Tasks & evaluation

WidowX generalization evaluation (314 trials per approach); absolute success rate +28% vs OpenVLA

Robot

WidowX(Bridge)

Inference

Inference device & speed

bf16 16 GB, 4-bit about 5 GB (README); Hz not disclosed; 5-step freeze +24%, async +40% speed

Inference memory

bf16 16 GB · 4-bit about 5 GB (README)

Weights bf16 (calc.)

14 GB

Where it runs

Not disclosed

Reinforcement learning & base

Reinforcement learning

Not used · Supervised learning including reasoning text

Base control

Fixed · No base control (WidowX arm)

Notes

Key features

Generates concrete reasoning such as plans·subtasks·object locations before actions, raising OpenVLA's absolute success rate by 28%. Tokens per step grow from 7 to about 350.

Notes

The reasoning annotation pipeline (Prismatic-7B, Grounding DINO, OWLv2, SAM, Gemini 1.0) took 7 days. The full paper does not state GPUs or batch size.

Verification

Confirmed