ACT-Real-PickOrange — SO-101 真机抓橙子(sim-warm-start,ACT 真机 best)
真机 SO-101 pick-orange 的 ACT 策略 best checkpoint:先在仿真(Isaac Lab LeIsaac-SO101-PickOrange-v0)训到 sim leaderboard best,再用 wsagi/leisaac-real-pick-orange(30 集真机遥操演示)warm-start 续训 5000 步(1.6 epoch)。
ACT policy for real-world SO-101 pick-and-place: warm-started from our sim-best ACT checkpoint, then fine-tuned for 5000 steps (1.6 epochs) on 30 real teleoperation episodes. Sim-warm-start is confirmed effective on the real arm — nearly 2× the from-scratch baseline.
真机结果 / Real-arm results(2026-07-12)
每轮场景放 3 只橙子,记每轮放入盘中的橙子数。Each round has 3 oranges on the table; we count oranges placed on the plate per round.
| 策略 / Policy | 轮次成绩 / Per-round | 均值 / Mean |
|---|---|---|
| 本模型(simwarm 5000)/ this model | 2, 1, 1, 3, 2, 1, 0, 1, 2, 2(10 轮) | 1.5 只/轮 |
| 从零 ACT 6000 / from-scratch baseline | 2, 0, 1, 1, 0(5 轮) | 0.8 只/轮 |
- simwarm 放入 ≥1 只的轮 9/10,含一轮 3 只满放;从零版仅 3/5。
- 结论:基于仿真 best 续训(sim-warm-start)有效 — 开环 MSE 低 ~40% 的增益在真机变现,均值近 2 倍。同数据的 FlowDP / GR00T-N1.7 头只能抓不能稳放,判别头(ACT)最强。
场景 / Scene
顶部相机视角(真机布置)/ top-camera view of the real setup:
配方 / Recipe
| 项 | 值 |
|---|---|
| 架构 / Arch | ACT(lerobot,chunk_size=100,n_action_steps=100) |
| init | sim-best ACT(Isaac Lab 仿真 leaderboard best,20k 步) |
| 数据 / Data | wsagi/leisaac-real-pick-orange 30 集 / 25,091 帧 @30fps |
| 续训 / Fine-tune | 5000 步(1.6 ep),lr 1e-5,batch 8,MEAN_STD + ImageNet norm |
| 归一化 / Norm stats | lerobot 0.5.2 warm-start 自动按真机数据集重建(sim stats 不残留) |
| 观测 / Obs | front + wrist 相机 640×480 + 6 维关节位置 |
用法 / Usage
from lerobot.policies.act.modeling_act import ACTPolicy
policy = ACTPolicy.from_pretrained("wsagi/ACT-Real-PickOrange")
相关项目 / Related projects
- https://github.com/vitorcen/isaaclab-experience — 伞仓:多 VLA 基线训练/评测全记录
- https://github.com/vitorcen/LeIsaac-Training — LeIsaac:SO-101 PickOrange 训练 / benchmark / 真机脚手架