VLA GalleryAll modelsGallery

VLA Gallery / All models / Reference baselines (pre-2024)

GR-1

ByteDance Research · 2023-12

VLAReference baselines (pre-2024)195Mregression

Pretrained by future-frame prediction on large-scale human video, then jointly learns actions and future images on robot data.

View architecture diagram in the galleryPaper ↗

Paper

Unleashing Large-Scale Video Generative Pre-training for Visual Robot ManipulationarXiv 2312.13139

Overview

Category
Reference baselines (pre-2024)
Type
VLA
Released
2023-12
Organization
ByteDance Research
Open weights
Partial
Parameters
195M (46M trainable)

Architecture

Backbone LLM
None (CLIP text encoder)
Vision encoder
MAE ViT + perceiver resampler
Action head
MLP action head + future frame prediction (regression)
Chunk
Not disclosed
Control rate
Not disclosed
Architecture (original summary)
MAE-pretrained ViT + perceiver resampler, CLIP text encoder; GPT-style causal transformer (12 layers·12 heads·hidden 384); 3-layer MLP action head + future image prediction, input length 10, future frame interval 3

Training

Pretraining accelerators
Not disclosed
Pretraining duration
Not disclosed
Pretraining steps & batch
50 epochs, batch 1024 (warmup 5 epochs, lr 3.6e-4, AdamW, cosine)
GPU/TPU-hours
Not disclosed
Fine-tuning
GPU and time not disclosed; CALVIN 20 epochs, batch 512, lr 1e-3, warmup 1 epoch
Data type
Human video; simulation·synthetic; real-robot demonstrations (collection method not stated)
Data scale
Pretraining Ego4D 800K clips·8M frames; CALVIN 20k+ expert trajectories (1% language-labeled); real-robot transport 1,775 demos, drawer 2,856 trajectories
Main datasets
Ego4D; CALVIN; in-house Kinova data
Tasks & evaluation
CALVIN ABCD→D success rate 94.9% (previous 88.9%), average length 4.21 (previous 3.06); ABC→D 85.4% (previous 53.3%); first task with 10% data 0.778; real-robot transport seen 0.79 · unseen instances 0.73 · unseen categories 0.30, drawer 0.75
Robot
7-DoF Kinova Gen2 single arm; CALVIN simulation

Inference

Inference device & speed
Not disclosed
Inference memory
Not disclosed
Weights bf16 (calc.)
0.4 GB
Where it runs
Not disclosed

Reinforcement learning & base

Reinforcement learning
Not used · Video prediction pretraining + imitation learning
Base control
Fixed · No base control (Kinova single arm·CALVIN sim)

Notes

Key features
Pretrained by future-frame prediction on large-scale human video, then jointly learns actions and future images on robot data. A language-conditioned GPT-style policy without an LLM·VLM backbone that improved CALVIN performance.
Notes
The industry note classifies it as VLA (by language input), a different criterion from the note that puts RT-1 under the same conditions as VA. Full Ego4D is over 3,500 h, of which GR-1 used 800K clips. CALVIN ABCD-D·ABC-D fine-tuned checkpoints released (Apache-2.0).
Verification
Confirmed

Other models in this category