GR00T-N1.7 Frozen LLM — Spa-Bench Epoch 12
This is the GR00T-N1.7 checkpoint evaluated as the Frozen LLM condition in Spa-Bench. “Frozen LLM” means that the language-model parameters were held fixed; the policy continued to receive language instructions.
Model details
| Field | Value |
|---|---|
| Model repository | justintiensmith/groot_multi_gpu_v4 |
| Base model | nvidia/GR00T-N1.7-3B |
| Checkpoint | End of epoch 12; step 71,304 |
| Robot | SO-101 single-arm manipulator |
| Inputs | Fixed middle RGB, wrist RGB, six absolute joint positions, text instruction |
| Outputs | Six absolute joint-position targets |
| Action horizon | 16 |
| Adaptation | LLM frozen; visual, multimodal/projector, VLLN, and diffusion-action modules updated |
| Optimizer | AdamW, learning rate 1e-4, weight decay 1e-5 |
| Schedule | 5% warm-up, then cosine decay |
| Hardware and batch | Four NVIDIA GH200 GPUs; 24 samples per device, global batch 96 |
The embedded train_config.json pins the motion-trimmed training data to
justintiensmith/VLA_Reasoning_Training_Dataset_1200_Trimmed_Start_5_Frame@c1231a8.
That exact snapshot is a two-camera projection of the repository's five-camera
trimmed release and is preserved by the groot-frozen-training-snapshot tag.
The tabular trajectory file and middle/wrist videos are byte-identical between
the two snapshots; episode metadata differs because the unused camera keys were
removed. Both contain 1,200 episodes, 570,386 frames, and 321 instruction
strings.
The run records the base-model identifier but not its immutable source
revision. The current public GR00T base revision at archival review is
2fc962b973bccdd5d8ce4f67cc63b264d6886495; it must not be assumed to be the
unrecorded training revision.
The author-supplied original
train_groot_n17_frozen_llm.sh
launcher is archived with the thesis artifact. It records seed 42, four-GPU
training, the immutable dataset revision, component-freezing flags, checkpoint
cadence, and the historical environment paths used for this run.
Deployment processing
For both GR00T variants, non-gripper action dimensions used the causal filter
filtered = 0.25 × current + 0.75 × previous_filtered. Filter state was
initialized from the measured robot state and reset for every rollout. The
gripper command was excluded.
Physical evaluation
| Condition | Successes | Rate |
|---|---|---|
| Familiar/in-distribution spatial instructions | 81/120 | 67.5% |
| All withheld/OOD spatial configurations | 133/300 | 44.3% |
| Matched OOD subset | 50/120 | 41.7% |
| Matched direct-manipulation controls | 98/120 | 81.7% |
The matched-control gap was 40.0 percentage points. These are physical rollout results, not simulation metrics.
- Rollouts:
justintiensmith/Spa_Bench_Full_GR00T-N1.7_Frozen_LLM - Thesis artifact:
justintiensmith/Imperial-Thesis
Intended use and limitations
This release supports reproduction and analysis of the Spa-Bench experiment. Older project files may call it “vision-only”; that term is inaccurate because several non-language modules were updated. The motion-trimmed data and deployment smoothing are policy-specific choices, so this comparison is not a controlled architecture ablation.
Robot policies can move hardware unexpectedly. Use conservative motion limits, an accessible emergency stop, a clear workspace, and direct supervision. Do not deploy this checkpoint for unattended or safety-critical operation.
Citation
Please cite the completed Spa-Bench MSc report, the thesis artifact, and the GR00T-N1.7 work referenced in the report.