gr00t-n1_5-sponge-marker-frozenvlm-decord-absolute-10k
GR00T N1.5 finetuned on an SO-101 sponge/marker manipulation dataset. Intermediate checkpoint at step 10,000 of a 30,000-step run.
This is the first checkpoint in this project trained on correctly-aligned video
frames. All earlier sponge-marker checkpoints (the delta-* and frozenllm-*
families) were trained with the torchcodec video backend, which misaligned roughly
35% of frames. This run uses decord (NVIDIA's recommended backend for the H.264
data). Do not compare metrics against the earlier families as if they differ only by
recipe — the input data differed.
Recipe
Stock GR00T N1.5 finetuning defaults — the same configuration NVIDIA's LIBERO checkpoints use. Backbone frozen, action head tuned.
| Base model | nvidia/GR00T-N1.5-3B |
tune_llm | False (frozen) |
tune_visual | False (frozen) |
tune_projector | True |
tune_diffusion_model | True |
| Action space | absolute — no delta, no custom normalization |
| Data config | so101_configs:So101SpongeMarkerDataConfig |
| Video backend | decord |
| Embodiment tag | new_embodiment |
Hyperparameters
| Batch size | 32 |
| Max steps | 30,000 (this checkpoint: 10,000) |
| Learning rate | 1e-4 peak, cosine schedule |
| Warmup | ratio 0.05 (1,500 steps) |
| LR at this checkpoint | 7.96e-05 |
| Weight decay | 1e-5 |
| Adam betas | (0.95, 0.999) |
| Precision | bf16 |
| Gradient checkpointing | off |
| Dataloader workers | 12 |
| Hardware | 1× NVIDIA RTX PRO 6000 Blackwell (96 GB) |
| Throughput | ~2.15 it/s |
Data
sponge_marker_merged_video — LeRobot-format, merged from two SO-101 teleop
datasets (grab sponge → place bowl; grab marker → place cup).
| Episodes | 100 |
| Frames | 78,522 |
| Cameras | video.front, video.handeye (480×640) |
| State / action | single_arm (5 joints) + gripper (1) |
| Epochs at 10k steps | 4.07 |
Training metrics
| Steps | loss (mean) | grad_norm (mean) | grad_norm (max) |
|---|---|---|---|
| 0–2k | 0.0643 | 0.90 | 13.32 |
| 2k–4k | 0.0234 | 0.44 | 0.93 |
| 4k–6k | 0.0202 | 0.38 | 0.73 |
| 6k–8k | 0.0180 | 0.35 | 0.64 |
| 8k–10k | 0.0161 | 0.33 | 0.64 |
Gradients are stable. The only spikes are steps 10–40 during warmup; after step 2,000
the maximum grad_norm over 8,000 steps is 0.93. The heavy-tail gradient spiking seen
previously on delta-action targets (with adam_beta2=0.999) does not appear here, so
adam_beta2 was left at its default.
wandb: mkpongm-uci/gr00t-sponge-marker
⚠️ Evaluation status: not yet validated on hardware
No robot evaluation has been run on this checkpoint. Training loss alone cannot establish that this policy is useful.
On absolute actions, a policy can reach very low training loss by learning to echo the current joint state rather than predicting the next action — a shortcut that accounted for 94.9% of predicted variance in an earlier run on this dataset. The loss curve here (below 0.03 within 300 steps, then a slow grind) is consistent with both genuine learning and that shortcut; the metric cannot distinguish them. Treat the numbers above as evidence the run is healthy, not that it works.
Because of this, a later checkpoint is not automatically better than this one. A shortcut-fitted policy can degrade with additional steps, so 10k is worth evaluating alongside 30k rather than assuming the final checkpoint wins.
Usage
Serve with the plain absolute-action data config:
python scripts/inference_service.py --server \
--model-path mkpongm/gr00t-n1_5-sponge-marker-frozenvlm-decord-absolute-10k \
--data-config so101_configs:So101SpongeMarkerDataConfig \
--embodiment-tag new_embodiment
Requires Isaac-GR00T and the project's
so101_configs.py. No normalization changes are needed — unlike the delta-*
checkpoints, which require so101_delta_configs.py and a matching gripper
normalization mode.
Contents
Model weights only (2 safetensors shards + index), config.json, and
experiment_cfg/metadata.json (dataset statistics required for serving). Optimizer
state is not included, so this checkpoint is servable but not resumable.