danish-lm-400m-grpo-mixed3-v3-beta004-freshopt-best-step250
Weights-only best-of snapshot from a "fresh-optim restart" experiment: resumed from mixed3-v3-beta004 best1-step5500 with fresh Adam state (no optimizer resume). Wiping optimizer momentum unlocked ~+5pp IF on top of the already-strong base.
Same base config (β=0.004, 3-way mix, from v31 SFT), just fresh optim.
Full offline eval
| eval | v31 SFT | v3 step-5500 | freshopt-step250 |
|---|---|---|---|
| IFEval-DA p-strict | 21.2 | 32.7 | 38.0 |
| IFEval-DA p-loose | 22.0 | 34.0 | 39.1 |
| IFEval-DA i-strict | 35.2 | 48.8 | 54.0 |
| IFEval-DA i-loose | 35.8 | 49.9 | 54.9 |
| GSM8K pass@1 (n=1317) | 17.4 | 27.9 | 28.5 |
| SciQ open-Q (n=1000) | 13.5 | 14.1 | 14.3 |
| CIT-gen (n=720) | 29.9 | 29.2 | 29.2 |
| textman_summary chrF++ | 41.1 | 40.9 | 41.1 |
| textman_rewrite chrF++ | 46.5 | 48.2 | 48.0 |
| CITMC (n=720) | 48.2 | 47.8 | 48.1 |
| SciQ-MC (n=1000) | — | 58.7 | 58.5 |
| PIQA (n=100) | 53 | 57.0 | 57.0 |
| ARC-Easy chat-MC | 44.4 | 42.1 | 42.2 |
| ARC-Challenge chat-MC | 29.4 | 29.1 | 29.3 |
| ARC-Easy logp | 40.6 | 41.4 | 41.2 |
| ARC-Challenge logp | 27.5 | 26.8 | 26.7 |
| GPQA-Diamond chat-MC | — | 21.2 | 21.2 |
| GPQA-Diamond logp | — | 23.7 | 24.2 |
| IFBench-DA p-strict | — | 8.7 | 8.0 |
| IFBench-DA i-strict | — | 9.0 | 8.7 |
Composite (from in-training callback, all full-test): 2.183 (step-5500 was 2.101).
Load
from transformers import AutoModelForCausalLM, AutoTokenizer
REPO = "jensjepsen/danish-lm-400m-grpo-mixed3-v3-beta004-freshopt-best-step250"
model = AutoModelForCausalLM.from_pretrained(REPO)
tok = AutoTokenizer.from_pretrained(REPO)
Chat: <|user|>{q}<|end|><|assistant|>, stop on <|end|>.
Weights only.