Qwen3-4B-Base + GRPO on MATH (boxed prompt), checkpoint 300
Qwen/Qwen3-4B-Base after 300 steps of GRPO (RL with a verifiable correctness reward) on the
MATH training split, with a rank-64 LoRA merged into the base weights. Plain HF model directory;
no chat template is needed or intended -- evaluate it as raw text under the boxed prompt.
Recipe
| prompt | Problem: {q}\nPlease reason step by step, and put your final answer within \boxed{}.\nSolution: |
| data | MATH train, 7,393 problems (MATH-500 excluded), no difficulty filtering |
| reward | binary correctness of the last \boxed{} (math-verify); truncated rollouts get no reward and no gradient |
| algorithm | GRPO, 32 prompts x 16 rollouts per step, T=1.0, no KL, unnormalised advantages, on-policy (1 iteration), DAPO-style token-level loss |
| lr / steps | 1e-5 constant, 300 steps (153,600 rollouts) |
| adapter | LoRA r=64, alpha=128, on q/k/v/o/gate/up/down projections, merged here into bf16 |
| cap | 4,096 completion tokens during training |
| stack | TRL 1.10, vLLM 0.26, transformers 5.15, PEFT 0.20; 8 x A100 80 GB |
Note: the trainer held the frozen base in fp32 during training (TRL's default); the merge adds the LoRA delta to the bf16 base weights.
Held-out MATH-500 (500 problems x 8 rollouts, T=0.6, top_p 0.95, 8,192-token cap, boxed prompt)
| step | 300 (this model) |
|---|---|
| pass@1 | 0.767 |
At step 300: commit 0.955, P(correct | committed) 0.803, pass@8 0.904. The base model with a typed
opening in the same harness: " To determine" 0.707, " Alright," 0.799.
AIME 2025 (30 x 32, 16,384-token cap), this model, no prefix: pass@1 0.114, pass@32 0.333.
During training the share of rollouts opening with To rose from 23% to 57%; Alright-type
openings from 0.2% to 1.6%. Every rollout of the run (text, reward, opener) was logged.