GR00T-N1.7-3B-Pick-Banana-v2
A fine-tuned version of nvidia/GR00T-N1.7-3B for fruit pick-and-place on a SO-ARM 101, trained on the jinseonylee/SO101_PickAndPlace_Fruit dataset.
Model Description
GR00T-N1.7 (Gr00tN1d7) is a vision-language-action (VLA) model for robot manipulation. This checkpoint is fine-tuned for a pick-and-place task on a SO-ARM 101 arm.
- Architecture: Gr00tN1d7 with Cosmos-Reason2-2B (Qwen-based) vision-language backbone + diffusion policy action head
- Base model: nvidia/GR00T-N1.7-3B
- Task: Pick and place fruit
- Robot: SO-ARM 101
- Action horizon: 16 steps
- Model dtype: bfloat16
Fine-tuning Configuration
| Parameter | Value |
|---|---|
| Tuned components | Diffusion model, projector (vision encoder + LLM backbone frozen) |
| Training steps | 13000 |
| Global batch size | 32 |
| Learning rate | 1e-4 (cosine, warmup ratio 0.05) |
| Weight decay | 1e-5 |
| State dropout prob | 0.2 |
| Optimizer | adamw_torch |
| Precision | bf16 |
| Final training loss | ~0.024 (avg train_loss 0.060) |
| Action representation | single_arm relative, gripper absolute |
| Attention | Flash Attention 2 |
Training Details
- Dataset: jinseonylee/SO101_PickAndPlace_Fruit — 102 episodes, ~78k frames (30fps), LeRobot v2.1 format
- Embodiment: SO-ARM 101 (
new_embodiment) - Max steps: 13000 (≈5.4 epochs over ~76.6k preloaded frames at batch 32)
- Final loss: 0.0244 at step 13000; train_runtime ~3h14m on a single GPU
- Reference: hyperparameters follow hi-space/GR00T-N1.7-3B-Pick-Orange, scaled from 6000 steps to keep a comparable epoch count for this larger dataset
Embodiment & Modalities
State inputs:
single_arm— arm joint positions (relative)gripper— gripper position (absolute)
Action outputs: same as state inputs (16-step horizon)
Cameras: front, wrist
Language conditioning: annotation.human.task_description
Usage
Refer to the NVIDIA Isaac GR00T repository for full inference and deployment instructions. Open-loop evaluation:
uv run python gr00t/eval/open_loop_eval.py \
--dataset-path <path-to>/SO101_PickAndPlace_Fruit_lerobot \
--embodiment-tag NEW_EMBODIMENT \
--model-path <path-to>/checkpoint-13000 \
--traj-ids 0 --execution-horizon 16 --steps 400
Intended Use
This model is intended for SO-ARM 101 fruit pick-and-place tasks matching the training dataset's setup. It is not guaranteed to transfer zero-shot to different scenes, objects, or hardware without additional fine-tuning.
License
This model inherits the license from the base model nvidia/GR00T-N1.7-3B / nvidia/Cosmos-Reason2-2B. Please refer to NVIDIA's terms for usage restrictions.