SONIC-X2 — NVIDIA SONIC ported to AgiBot X2 Ultra
Whole-body motion-tracking policy for the AgiBot X2 Ultra humanoid (29 DOF, head-fixed), finetuned from the released NVIDIA G1 SONIC checkpoint. The bet is data diversity: the full BONES-SEED motion corpus retargeted to X2.
This checkpoint
model_step_011300.pt— training step 11,300 (in progress toward 20k).- Warm-started from the official G1 SONIC release with a 6-joint X2→G1 permutation fix (waist roll/pitch, wrist yaw/roll) applied to the seed.
- Three encoders trained jointly (SONIC-X2 joint-trajectory / SMPL / VR-teleop), shared FSQ token space, shared decoder.
Metrics
Official-protocol offline eval (128 random clips, terminations on = fail-and-truncate like the official evaluator; root-relative MPJPE over pre-termination frames):
| step | MPJPE (official protocol) | Success rate | notes |
|---|---|---|---|
| 12.8k | 38.3 mm | — | |
| 14k | 38.3 mm (median 35) | 35 % (45/128) |
Official SONIC target: MPJPE < 30 mm, success > 0.98, reached at ~100k steps on 128 GPU. This run is at ~14k steps. The gap is dominated by global root-position tracking during locomotion: 78 % of failures are root drift (robot lags the reference velocity and exceeds the 0.25 m anchor threshold). Success rate by motion type at 14k: idle/gesture 75 %, but walk/run/turn ≈ 0 % — pose accuracy (MPJPE 38 mm) is already decent; holding global position across a full locomotion clip is the hardest part and converges last.
MPJPE-trend on the earlier 8-clip probe (biased toward dynamic clips, terminations off): 59.3 → 56.0 → 50.9 → 48.8 → 47.6 → 46.0 → 44.2 mm (1k → 12.7k), monotonic.
Training config
- Corpus: 122,572 clips (61,677 unique + mirrors), GMR-retargeted to X2.
- Physics: per-joint efforts transcribed from the X2 Ultra URDF; KP/KD from the
AgiBot SOP §3.1
mimicprofile, except wrist KD tuned 2.0→1.0 to match G1's tracking-lag coefficient (deploy must mirrorkd=1.0on the wrists). - 32–64 GPU multi-node (Beaker), PPO with KL-adaptive LR, adaptive motion sampling, save every 100 steps.
Load
import torch
sd = torch.load("model_step_011300.pt", map_location="cpu")["policy_state_dict"]
config.yaml / meta.yaml carry the full Hydra config needed to instantiate
the actor-critic.