Diffusion policy — SO-101 pick block into bowl
Diffusion Policy trained with LeRobot on a real SO-101 leader/follower arm to perform the task "pick up the block and place it in the bowl".
- Dataset: sahilapage/so101_pick_block_bowl_20260718_211257 — 50 teleop episodes, 32,504 frames, 30fps, dual camera (wrist + external).
- Training: 180,000 steps, batch size 8, lr 1e-4 peak with cosine schedule (500 warmup steps,
decaying to ~0 by step 200k), on an RTX 4090. Final training loss ≈ 0.003-0.004. This checkpoint
is the final saved step (180000, the last save point at/before completion —
save_freq=20000). - Hardware: verified with a real-hardware rollout — successfully picked up the block and placed it in the bowl on the SO-101 follower arm. Much slower per-action than the ACT policy (iterative denoising per action tick vs. ACT's single forward pass), especially on CPU-only inference.
Use
from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy
policy = DiffusionPolicy.from_pretrained("sahilapage/diffusion_so101_pick_block_bowl")
Or for a live rollout on SO-101 hardware:
lerobot-rollout --strategy.type=base --policy.path=sahilapage/diffusion_so101_pick_block_bowl \
--robot.type=so101_follower --robot.port=<your follower port> --robot.id=so101_follower \
--robot.cameras="{ wrist: {type: opencv, index_or_path: <wrist cam>, width: 640, height: 480, fps: 30}, external: {type: opencv, index_or_path: <external cam>, width: 640, height: 480, fps: 30}}" \
--task="pick up the block and place it in the bowl" --duration=300 --fps=30
See also
- ACT policy trained on the same task/dataset: sahilapage/act_so101_pick_block_bowl