act-so101-pickplace-dr — domain-randomization-trained ACT
ACT (~52M) trained from scratch on
so101-sim-pickplace-dr —
150 successful SO-ARM100 pick-and-place demonstrations recorded under per-reset
visual + physics domain randomization in MuJoCo. Same recipe as the nominal
baseline (chunk 45, n_action_steps 15, batch 8, 60k steps); only the data
differs, so every improvement below is attributable to the data.
Results
Robustness sweep, 15 eval episodes per randomization level, fixed eval seed:
| nominal | + visual DR | + physics DR | + full DR | |
|---|---|---|---|---|
| Scripted expert (privileged state) | 100% | 100% | 100% | 100% |
| ACT v1 (trained on nominal data) | 60% | 60% | 53% | 40% |
| ACT-DR (this model) | 87% | 87% | 87% | 100% |
Standard 20-episode nominal protocol: 18/20 (90%) — vs the nominal-trained baseline's 65% (75% with temporal ensembling). Training on randomized data was not a robustness tax: it improved nominal performance by 27 points while staying flat across every distribution shift.
Measurement integrity note: the companion project's originally-published claim that the nominal policy "collapses to 0%" under randomization was an eval-harness artifact (a GL-context teardown bug fed the policy black frames); it was caught, fixed, and re-measured — full forensic in the sim2cell README.
Training
- LeRobot 0.6.0
lerobot-train, single Kaggle T4, AMP — 60k steps in 2h12m (~0.13 s/step), loss 9.39 → 0.036 (28.7 epochs of 150 episodes) --policy.type=act --policy.chunk_size=45 --policy.n_action_steps=15 --batch_size=8
Use
from lerobot.policies import make_pre_post_processors
from lerobot.policies.act.modeling_act import ACTPolicy
repo = "ahmedsohail2003/act-so101-pickplace-dr"
policy = ACTPolicy.from_pretrained(repo).to("cuda").eval()
pre, post = make_pre_post_processors(
policy_cfg=policy.config, pretrained_path=repo,
preprocessor_overrides={"device_processor": {"device": "cuda"}},
)
# batch: observation.state (1,6) + observation.images.front / .wrist (1,3,224,224) in [0,1]
action = post(policy.select_action(pre(batch)))
Provenance
Data recorded by a scripted expert (success-filtered) in the author's MuJoCo
work-cell; robot model from
MuJoCo Menagerie
(trs_so_arm100, Apache-2.0).