smolvla-so101-pickplace-v2 — 90% language-conditioned pick-and-place
SmolVLA (450M) fine-tuned on
so101-sim-pickplace-v2
(160 SO-ARM100 episodes incl. 12.5% verified-retry recovery demonstrations):
"Pick up the red block and place it in the blue tray."
Results (20 eval episodes, fixed seed disjoint from training)
| Policy | Success | |
|---|---|---|
| SmolVLA base, zero-shot | 0% | |
| SmolVLA on 100 nominal eps | 55% | all failures = right-side non-engagements (coverage) |
SmolVLA on the 160-ep -v2 set (this model) | 90% | coverage fixed; one eval episode shows a live miss→retry→success (165 steps vs the stereotyped ~105) |
ACT (~52M specialist BC) on the same -v2 data | 50% | hurt by the same data — see below |
The cross-architecture finding
The identical -v2 dataset moved two architectures in opposite directions:
Trained on -v2 (160 eps, incl. recovery branches) | vs. its nominal-data baseline |
|---|---|
| ACT (chunked L1 regression) | 65% → 50% (and temporal ensembling flips from +10 to −15) |
| SmolVLA (flow-matching action expert) | 55% → 90% |
Recovery demonstrations make the data behaviorally multimodal: from near-identical states, some demonstrations proceed to lift while others re-approach. Chunk regression averages incompatible continuations into compromise actions; a flow-matching generative head simply learns both modes. Full analysis: the ACT side is documented as a negative-result model card.
Fine-tuning recipe (single Kaggle T4, free tier)
LeRobot 0.6.0 lerobot-train: batch 8, 12k steps (~8 h), AMP, frozen VLM /
action-expert-only (defaults), cameras renamed to the base's naming
(front→camera1, wrist→camera2). Same recipe as the
55% v1 model —
only the data changed.
Provenance
Base lerobot/smolvla_base (Apache-2.0);
author-recorded scripted-expert demonstrations in MuJoCo; robot model from
MuJoCo Menagerie (trs_so_arm100).