SmolVLA - Clutter (joint, 2-cam)
HuggingFace LeRobot SmolVLA fine-tuned on the ManiGuard clutter base task (sim Franka Panda). Part of the ManiGuard VLA benchmark - SmolVLA vs pi0.5 vs GR00T on the same task families with identical data, cameras, and controller.
Model
- Base: lerobot/smolvla_base - SmolVLM2 vision-language backbone + flow-matching action expert
- Embodiment: Franka Panda, 8-D joint state/action (7 arm joints + 1 gripper), padded to SmolVLA's 32-D
- Cameras (2):
observation.images.top(overview =leftview) +observation.images.wrist(256x256) - Action: absolute joint targets (fed straight to a JointController at eval); 50-step chunk
- Tuning: SmolVLA default - vision encoder frozen, train the action expert (no LoRA)
Training
- 8-GPU accelerate DDP, global batch 512, 3522 steps (~2 epochs over 901,520 frames)
- Data: IDEAS-Lab-Northwestern/datagen-clutter-v1-joint-5cam - the 5-cam datagen dataset, prepared to a 2-cam standard-keyed copy (H.264 videos)
Usage
Load with SmolVLAPolicy.from_pretrained("IDEAS-Lab-Northwestern/smolvla-base-datagen-v1-clutter-joint-2cam-yanZ") from LeRobot. The checkpoint carries the normalization stats.
WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets) + 2 cameras (
observation.images.top= theleftoverview,observation.images.wrist). A mismatched controller, camera set, or overview view silently feeds an out-of-distribution input.
Paper & Citation
Part of ManiGuard: paper (arXiv:2608.17386) · code · docs
@misc{peng2026maniguard,
title = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
Safety Evaluation and Improvement of Robotic Manipulation},
author = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
and Zhang, Ruohan and Zhu, Qi},
year = {2026},
eprint = {2608.17386},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2608.17386},
}
License
The base model lerobot/smolvla_base
is released under the Apache License, Version 2.0. ManiGuard's fine-tuning
contributions are also released under Apache-2.0.