g1-inspire-piston-starvla-gr00t-n1d7

A GR00T-N1.7 fine-tune for a Unitree G1 (29-DoF) with Inspire hands performing a piston pick-and-place task, trained inside starVLA with the CosmosGR00TN1d7 framework.

Unlike a from-scratch action head, this model is initialised directly from nvidia/GR00T-N1.7-3B — all 1031 checkpoint tensors (VLM backbone and flow-matching action head) load 1:1 — and then fine-tuned end-to-end on the piston dataset.

Architecture (~3.1B params)

There are no "future tokens"; the DiT contains no cross-attention blocks in the classic sense — VL conditioning enters via the alternating cross-attend pattern.

Task & I/O contract

• Instruction: "pick up the piston."
• Observation: single ego (head) camera view at 256×256 + language. Proprioceptive state is supported but was trained with heavy dropout (see below).
• State: 29-dim upper body [left_arm 7 | right_arm 7 | left_hand 6 | right_hand 6 | waist 3]
• Action: 30-dim absolute joint targets (radians) [left_arm 7 | right_arm 7 | left_hand 6 | right_hand 6 | base_height 1 | navigate_command 3]
• Action horizon: 30 steps per inference (receding horizon)
• Embodiment slot: 25 — GR00T's pretrained unitree_g1_full_body_with_waist_height_nav_cmd, whose action layout matches this robot, so the category-specific projectors warm-start rather than starting cold.
• Normalisation: q99 on both state and action (stats in dataset_statistics.json). Real 30-dim action / 29-dim state are zero-padded to the head's 132-wide projectors at runtime; the loss is masked back to the real dims.

Training

• Data: birbirll/g1-inspire-piston-pick-place (LeRobot v2.1, 102 episodes, 50 fps, single 240×424 ego view)
• Steps: 10,000 · effective batch 32 (micro-batch 8 × grad-accum 4)
• Full fine-tune: backbone and head trainable (tune_llm=true, tune_visual=true)
• Per-module LRs: backbone 1e-5, action head 1e-4, cosine schedule with min LR 1e-6, 50 warm-up steps
• Optimizer: bitsandbytes PagedAdamW8bit, betas (0.9, 0.95), wd 1e-8, grad clip 1.0
• Augmentation: ego-view color jitter (brightness 0.3 / contrast 0.4 / saturation 0.5 / hue 0.08)
• state_dropout_prob = 0.8 — proprioceptive state is available but zeroed 80% of the time during training, so the policy is strongly camera-driven and does not lean on proprio. This matters for sim/real transfer: a state-reliant policy can minimise training loss while ignoring the image, then fail to correct visually in closed loop.
• Plain accelerate (no DeepSpeed), gradient checkpointing, single GPU.

Files

config.yaml                       # resolved training config (required to load)
dataset_statistics.json           # q99 norm stats (required to un-normalise actions)
final_model/pytorch_model.pt      # trained weights (~8.9 GB)
starvla_gr00t_n1d7_piston.yaml    # training config as authored
run_starvla_GR00T_N1d7_piston_train.sh  # exact launcher used
config.full.yaml, summary.jsonl   # provenance

config.yaml and dataset_statistics.json must sit one level above the checkpoint file — starVLA's loader resolves them from the run root.

Usage

Requires starVLA with the CosmosGR00TN1d7 framework, plus nvidia/Cosmos-Reason2-2B available locally (it is a gated repo — request access first).

Serve it over the GR00T ZMQ protocol:

python deployment/model_server/server_policy_gr00t_zmq.py \
  --ckpt_path <run_dir>/final_model/pytorch_model.pt \
  --port 5555 --use_bf16 --no_state \
  --fallback_instruction "pick up the piston."

--no_state serves it camera-only, which is well matched to the 0.8 state dropout and avoids depending on the client's proprio format. Drop the flag to feed real state.

For closed-loop control, a GR00T-protocol client (e.g. a WBC bridge) should use the UNITREE_G1 contract — absolute joints, single ego frame, 30-step chunks — not the "REAL_G1 / N1.7" contract (relative arms, 2-frame video, 40-step), despite this model's N1.7 architecture. The protocol and the architecture are independent choices here.

Limitations

• Trained on a single task from 102 episodes in one scene; it is not a general-purpose manipulation policy.
• Validated in Isaac Lab simulation. Real-robot behaviour is untested.
• Actions are absolute joint targets and must be executed at the demonstration rate (50 fps in sim time). Replaying them faster (e.g. pacing by wall-clock while the simulator runs slower than real time) causes large overshoot.
• Camera intrinsics/FOV matter: the policy expects an ego view matching the training capture (D435-like, 240×424 aspect, resized to 256²). A different FOV is out-of-distribution.

Attribution & licensing

Derived from NVIDIA GR00T-N1.7 (nvidia/GR00T-N1.7-3B) and NVIDIA Cosmos-Reason2-2B, and contains fine-tuned weights of both. The DiT and embodiment-conditioned MLP modules in starVLA are vendored from Isaac-GR00T (Apache-2.0). Use of these weights is subject to NVIDIA's licence and model-access terms for the upstream models in addition to anything stated here.