danish-lm-400m-grpo-mixed-v1-best2-step4375
Rank-2 best-of-3 snapshot from a GRPO run that interleaved GSM8K + IFEval-DA (combined our-46 + Google-schema constraints) as a per-example dispatched reward. Base: jensjepsen/danish-lm-400m-sft-v31-avg-top3.
Training
- Reward: per-example dispatch —
reward_gsm8kon gsm8k rows,reward_ifeval_combined(our 46 constraints + Google IFEval schema) on IF rows. - Data:
jensjepsen/danish-if-grpo-combined-v1(10k) interleaved withjensjepsen/danish-gsm8k:sft:train(~7.5k), 50/50 mix. - Resumed from
jensjepsen/danish-lm-400m-grpo-if-combined-v1-step2400(combined-run peak) at step 2400 into an already-in-progress mixed run; current model is at step 4375 of that mixed schedule. - Snapshot ranking: composite score =
ifeval-da prompt-strict + inst-strict + gsm8k pass@1(each ∈ [0, 1]). Best-K tracker inscripts/train_grpo_verifier.py(top-3 rolling, model+tokenizer only, no optimizer).
Evaluation (offline, fp16)
Baseline column is jensjepsen/danish-lm-400m-sft-v31-avg-top3 (from its README).
IFEval-DA (danish-foundation-models/ifeval-da, n=539)
Uses our patched Google verifier chain — DFM renamed change_case:capital_letters / change_case:lowercase_letters / detectable_format:constrained_response_with_argument without publishing verifier code; patches document divergence.
| metric | base | step 4375 | Δ |
|---|---|---|---|
| prompt-strict | 21.2 | 28.2 | +7.0 |
| prompt-loose | 22.0 | 29.9 | +7.9 |
| inst-strict | 35.2 | 44.9 | +9.7 |
| inst-loose | 35.8 | 46.1 | +10.3 |
Generation
| eval | base | step 4375 | Δ |
|---|---|---|---|
| GSM8K 0-shot pass@1 (n=1317) | 17.39 | 25.89 | +8.50 |
| SciQ open-Q pass@1 (n=1000) | 13.50 | 13.60 | +0.10 |
| CIT-gen (n=720) | 29.86 | 28.5 | -1.36 |
| textman_summary chrF++ | 41.11 | 40.84 | -0.27 |
| textman_rewrite chrF++ | 46.51 | 46.80 | +0.29 |
Multiple-choice (chat-MC)
| eval | base | step 4375 | Δ |
|---|---|---|---|
| CITMC (n=720) | 48.19 | 50.14 | +1.95 |
| SciQ-MC (n=1000) | — | 59.5 | — |
| PIQA (n=100 full test) | 53 (n=50) | 59.0 | +6 |
ARC-Easy chat-MC (jensjepsen/danish-arc:arc_easy:test) | 44.40 | 42.55 | -1.85 |
ARC-Challenge chat-MC (jensjepsen/danish-arc:arc_challenge:test) | 29.35 | 28.92 | -0.43 |
Multiple-choice (raw log-prob)
| eval | base | step 4375 | Δ |
|---|---|---|---|
| ARC-Easy logprob (n=2376) | 40.61 | 40.07 | -0.54 |
| ARC-Challenge logprob (n=1172) | 27.47 | 27.56 | +0.09 |
Load
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("jensjepsen/danish-lm-400m-grpo-mixed-v1-best2-step4375")
t = AutoTokenizer.from_pretrained("jensjepsen/danish-lm-400m-grpo-mixed-v1-best2-step4375")
Chat format (single-turn): <|user|>{q}<|end|><|assistant|>, stop on <|end|>.
Provenance
- Base SFT: danish-lm-400m-sft-v31-avg-top3
- GRPO combined intermediate (start of mixed): step2400
- Sibling best-of:
best1-step3375,best3-step3250 - Training set: danish-if-grpo-combined-v1 + danish-gsm8k
Weights only (no optimizer/scheduler) — for eval/inference.