pi0.5 AssembleBench - CG-DAgger round 3 (final)

The final checkpoint from the writeup: the BC baseline plus three rounds of code-gated DAgger. This is what examples/run_eval.py loads by default.

pi0.5 finetuned for AssembleBench, a 14-task contact-rich robot assembly benchmark on the NIST ATB-1 taskboard: insert round and square pegs, mesh gears, thread nuts. Built on Isaac Lab Arena with the DROID robot platform (Franka Panda + Robotiq 2F-85).

Read the writeup: Benchmarking Robot Models on Contact-Rich Assembly.

Usage

git clone https://github.com/hud-evals/assemble-bench.git
cd assemble-bench
pip install -r examples/requirements.txt

# Serve the simulator (see the repo README for the Isaac Sim install)
OMNI_KIT_ACCEPT_EULA=YES python -m hud.environment.server env.py --port 8765

# Evaluate this checkpoint
python examples/run_eval.py --task peg_round_16mm --checkpoint hud-evals/pi05-AssembleBench-cgdagger-r3

The DROID input/output convention is not baked into the checkpoint. examples/pi05_agent.py carries the adapter: front and wrist RGB resize-padded to 224, an 8-D state of 7 joint positions plus gripper, and 8-D absolute joint-target actions at 15 Hz with a binarized gripper.

Training

Finetuned from DAVIAN-Robotics/pi05_droid_jointpos (full finetune, VLM included) on hud-evals/AssembleBench, 1355 synthetic demonstrations generated by scripted experts reading privileged simulator state, with no teleoperation.

Code-gated DAgger (CG-DAgger): the policy drives, code detects failure and hands control to a scripted expert that demonstrates a recovery, and the policy is retrained on those intervention segments. It is HG-DAgger with the human replaced by a failure gate. Corrections were collected only on peg_round_8mm, so the other round-peg sizes measure whether the recovery transfers.

Three rounds were collected. Round 1 targeted grasp failure (grip lower and reattempt), round 2 off-center peg placement (re-center before inserting), and round 3 insertion failure from releasing the grasp mid-insertion (insert without ever opening the gripper). Later rounds improved the looser clearances (12 mm and 16 mm); 4 mm stayed flat.

Limitations

Task-specific: this is not a generalist checkpoint, and it is trained purely in simulation. Success on these tasks is sparse - a handful of wins out of thirty on the harder families - so the writeup reports dense reward alongside binary success. Square pegs and small gears remain close to zero.

Related checkpoints