Chain-of-Action

ByteDance Seed; University of Adelaide; NUS; CAS; CSIRO · 2025-06

VAVANot disclosedregression

Generates the trajectory backward from the goal keyframe to the current time, reflecting the global goal in actions.

Paper

Backbone LLM: None
Vision encoder: ResNet-18 (multi-view)
Action head: Reverse autoregressive continuous actions starting from keyframes (regression)
Chunk: Not disclosed
Control rate: Not disclosed
Architecture (original summary): Multi-view ResNet-18 + 4-layer transformer encoder·7-layer decoder; reverse-order autoregressive trajectory generation starting from keyframe actions, continuous action tokens·latent consistency loss, multi-token prediction heads during training, dynamic stopping

Pretraining accelerators: N/A
Pretraining duration: N/A
Pretraining steps & batch: N/A
GPU/TPU-hours: Not disclosed
Fine-tuning: Per task: 1× H100, 20,000 iterations, batch 128 (same as README default); time Not disclosed
Data type: Simulation·synthetic; real-robot teleoperation
Data scale: RLBench 100 demos per task; real 35–81 demos per task
Main datasets: RLBench; in-house Fetch data
Tasks & evaluation: Centered on 60 RLBench tasks; 8 real Fetch kitchen tasks
Robot: Fetch (real); RLBench simulation

Reinforcement learning: Not used · Imitation learning (behavior cloning) only
Base control: Fixed · Base movement uses Fetch's built-in LiDAR navigation (separate stack); policy outputs absolute EE pose only

Key features: Generates the trajectory backward from the goal keyframe to the current time, reflecting the global goal in actions. Trained on 1× H100 per task.
Notes: Reports improvement over ACT · DP, but the note gives no figures.
Verification: Confirmed