VLA GalleryAll modelsGallery

VLA Gallery / All models / VA

RISE

Shanghai Noematrix Intelligence Technology; Shanghai Jiao Tong University · 2024-04

VAVANot discloseddiffusion

Encodes a single-view point cloud with sparse 3D convolutions and outputs actions via a transformer and diffusion head.

View architecture diagram in the galleryPaper ↗

Paper

RISE: 3D Perception Makes Real-World Robot Imitation Simple and EffectivearXiv 2404.12281

Overview

Category
VA
Type
VA
Released
2024-04
Organization
Shanghai Noematrix Intelligence Technology; Shanghai Jiao Tong University
Open weights
No
Parameters
Not disclosed

Architecture

Backbone LLM
None
Vision encoder
Sparse 3D encoder (MinkowskiEngine)
Action head
CNN diffusion head (20 inference steps) (diffusion)
Chunk
20 steps
Control rate
Not disclosed
Architecture (original summary)
MinkowskiEngine sparse 3D encoder (5mm voxels, single RGB-D point cloud) + Transformer (4 encoder·1 decoder, d_model 512, d_ff 2048) + CNN diffusion head (100 training·20 inference steps); action horizon 20

Training

Pretraining accelerators
N/A
Pretraining duration
N/A
Pretraining steps & batch
N/A
GPU/TPU-hours
Not disclosed
Fine-tuning
Per task: 2× A100, batch 240, lr 3e-4, warmup 2000 steps; time·epochs Not disclosed
Data type
Real-robot teleoperation
Data scale
50 demos per task
Main datasets
In-house Flexiv teleoperation
Tasks & evaluation
Collect Cups, Collect Pens, Pour Balls, Push Block, Push Ball, Stack Blocks; compared with ACT, Diffusion Policy, Act3D, DP3
Robot
Flexiv Rizon arm + Dahuan AG-95 gripper

Inference

Inference device & speed
Intel i9-10900K + RTX 3090 workstation; latency not disclosed
Inference memory
Not disclosed
Weights bf16 (calc.)
Not disclosed
Where it runs
Local PC

Reinforcement learning & base

Reinforcement learning
Not used · Imitation learning (behavior cloning) only
Base control
Fixed · No base control (Flexiv arm)

Notes

Key features
Encodes a single-view point cloud with sparse 3D convolutions and outputs actions via a transformer and diffusion head. Compared with ACT·DP·Act3D·DP3 using 50 demonstrations per task.
Notes
Hybrid architecture of sparse convolution, transformer, and CNN diffusion head. The successor RISE-2 was released 2025-05 (README), not profiled.
Verification
Confirmed

Other models in this category