Pick-Cube SO-101 — SmolVLA policy
A SmolVLA policy finetuned to "grab the cube and put it in the box" on the SO-101 arm, from three camera views. Trained on the dobri420/pick-cube-so101 teleoperation dataset.
89% grab rate (72/81) on the real arm — the best of a coverage-vs-optimization sweep, and balanced across reach depth (25 / 20 / 15 cm = 89 / 93 / 85 %).
Training corpus vs. performance
Same workspace half-disk in both panels (rings = reach in cm from the shoulder-pan axis, radials = target angle), so coverage and success line up cell-for-cell. Left: where the training demos actually grasped (SO-101 forward kinematics on the jaw center). Right: where the resulting policy succeeds, measured on a held-out grid of 81 targets (27 per depth).
Training
- Base:
lerobot/smolvla_base(expert finetuned, VLM frozen). - Data: the first 340 episodes of
dobri420/pick-cube-so101(the five tranches through twist-CCW; the dataset has since grown to 440 with a twist-CW tranche not used here). - Schedule: 2.56M samples @ batch 64 (40k steps), bf16, on a single B200.
- Inputs: three 256×256 camera views (
camera1/2/3) + 6-DoF joint state. Output: 6-DoF action, 50-step chunks.
Usage
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
policy = SmolVLAPolicy.from_pretrained("dobri420/pick-cube-so101")
The repo ships the runnable inference bundle: model weights, config.json, and the
normalization pre/post-processor pipelines (the mean/std that map real joint units
↔ the model's normalized space live in the *_processor.safetensors — they are
required for inference). train_config.json records the full training recipe.
Built with LeRobot.