Nemotron-3-Nano-4B · 16k-token budget · grpo · firmbeetle
RL-finetuned nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 trained for math reasoning under a 16,384-token generation budget. Released as part of an anonymous ICLR 2027 submission.
Run ID (petname): firmbeetle · checkpoint step 120
Note: this run was stopped at step 120 of the planned 300; other checkpoints in the budget sweep are at step 300.
Training
| Base model | nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 |
| Algorithm | GRPO (leave-one-out baseline, group reward normalization, token-level loss). Responses that hit the budget are truncated and scored as-is. |
Generation budget (max_new_tokens) | 16,384 |
| Data | DeepScaleR (math), 3 epochs max |
| Batch | 32 prompts × 8 rollouts per step |
| Optimizer | Adam, cosine LR schedule, peak LR 5e-7, 10 warmup steps |
| Steps | 120 |
| Reward | binary answer correctness (\boxed{} extraction) |
| Weights dtype | F32 |
Training prompt (user turn, rendered with the base model's chat template):
Think step-by-step to solve the following problem. Output your answer inside of \\boxed{} tags.:
{problem}
Let's think step-by-step
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "budget-internalization-iclr2027/nemotron3-nano-4b-16k-grpo-firmbeetle-s120"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto", trust_remote_code=True)
With vLLM: vllm serve budget-internalization-iclr2027/nemotron3-nano-4b-16k-grpo-firmbeetle-s120 --trust-remote-code
License
Inherits the license of the base model (nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16).