danish-lm-400m-grpo-mixed-v1-best3-step3250
Rank-3 best-of-3 snapshot from a GRPO run that interleaved GSM8K + IFEval-DA (combined our-46 + Google-schema constraints) as a per-example dispatched reward. Base: jensjepsen/danish-lm-400m-sft-v31-avg-top3.
Training
- Reward: per-example dispatch —
reward_gsm8kon gsm8k rows,reward_ifeval_combined(our 46 constraints + Google IFEval schema) on IF rows. - Data:
jensjepsen/danish-if-grpo-combined-v1(10k) interleaved withjensjepsen/danish-gsm8k:sft:train(~7.5k), 50/50 mix. - Resumed from
jensjepsen/danish-lm-400m-grpo-if-combined-v1-step2400(combined-run peak) at step 2400 into an already-in-progress mixed run; current model is at step 3250 of that mixed schedule. - Snapshot ranking: composite score =
ifeval-da prompt-strict + inst-strict + gsm8k pass@1(each ∈ [0, 1]). Best-K tracker inscripts/train_grpo_verifier.py(top-3 rolling, model+tokenizer only, no optimizer).
Evaluation (offline, fp16)
Baseline column is jensjepsen/danish-lm-400m-sft-v31-avg-top3 (from its README).
IFEval-DA (danish-foundation-models/ifeval-da, n=539)
Uses our patched Google verifier chain — DFM renamed change_case:capital_letters / change_case:lowercase_letters / detectable_format:constrained_response_with_argument without publishing verifier code; patches document divergence.
| metric | base | step 3250 | Δ |
|---|---|---|---|
| prompt-strict | 21.2 | 29.3 | +8.1 |
| prompt-loose | 22.0 | 29.7 | +7.7 |
| inst-strict | 35.2 | 44.5 | +9.3 |
| inst-loose | 35.8 | 44.9 | +9.1 |
Generation
| eval | base | step 3250 | Δ |
|---|---|---|---|
| GSM8K 0-shot pass@1 (n=1317) | 17.39 | 24.60 | +7.21 |
| SciQ open-Q pass@1 (n=1000) | 13.50 | 13.80 | +0.30 |
| CIT-gen (n=720) | 29.86 | 29.2 | -0.66 |
| textman_summary chrF++ | 41.11 | 40.94 | -0.17 |
| textman_rewrite chrF++ | 46.51 | 47.20 | +0.69 |
Multiple-choice (chat-MC)
| eval | base | step 3250 | Δ |
|---|---|---|---|
| CITMC (n=720) | 48.19 | 48.75 | +0.56 |
| SciQ-MC (n=1000) | — | 59.6 | — |
| PIQA (n=100 full test) | 53 (n=50) | 59.0 | +6 |
ARC-Easy chat-MC (jensjepsen/danish-arc:arc_easy:test) | 44.40 | 42.55 | -1.85 |
ARC-Challenge chat-MC (jensjepsen/danish-arc:arc_challenge:test) | 29.35 | 28.33 | -1.02 |
Multiple-choice (raw log-prob)
| eval | base | step 3250 | Δ |
|---|---|---|---|
| ARC-Easy logprob (n=2376) | 40.61 | 40.45 | -0.16 |
| ARC-Challenge logprob (n=1172) | 27.47 | 27.47 | 0.00 |
Load
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("jensjepsen/danish-lm-400m-grpo-mixed-v1-best3-step3250")
t = AutoTokenizer.from_pretrained("jensjepsen/danish-lm-400m-grpo-mixed-v1-best3-step3250")
Chat format (single-turn): <|user|>{q}<|end|><|assistant|>, stop on <|end|>.
Provenance
- Base SFT: danish-lm-400m-sft-v31-avg-top3
- GRPO combined intermediate (start of mixed): step2400
- Siblings:
best1-step3375,best2-step4375 - Training set: danish-if-grpo-combined-v1 + danish-gsm8k
Weights only (no optimizer/scheduler) — for eval/inference.