Fixed-reference Nash (per-prompt weights)
Research checkpoint. Nash aggregation with a frozen reference policy instead of an adaptive one, with per-prompt multipliers.
Fine-tuned from Qwen/Qwen2.5-7B-Instruct. This repository is one of three arms trained and evaluated
under one protocol so that the only difference between them is the
aggregation rule; the other two are linked below.
Training setting
Retrained under the protocol of "Back to Blackwell" (PROSPER) on WildChecklists, where every prompt carries its own checklist items and the judge scores one item at a time.
| Prompts | 522 (union-filtered so all three arms train on one prompt set) |
| Learner pairs | 14,616 (all 28 unordered pairs of 8 sampled responses per prompt) |
| Pipeline epochs | 2 |
| Optimizer steps | 228 at batch size 128 |
| Learning rate | 3e-7, weight decay 1e-6, AdamW, warmup ratio 0.1, grad clip 1.0 |
| Max sequence / prompt length | 2048 / 1024 |
| Seed | 555134 |
| Candidate decoding | temperature 0.8, top_p 0.9, 8 responses per prompt |
| Judge for training signal | local open-weight judge, 10 judgments per pair averaged over both presentation orders |
Provenance recorded by the run: dataset manifest a0464bcb7c0d30d6, solver artifact not recorded.
Evaluation
Greedy generation (temperature 0) against each benchmark's released baseline answers, judged by a local open-weight judge at temperature 0 in both presentation orders with ties counted as 0.5.
| Arm | Arena-Hard | AlpacaEval |
|---|---|---|
| NBPO | 0.6333 | 0.4124 |
| PROSPER | 0.6225 | 0.4071 |
| Fixed-reference Nash | 0.6111 | 0.4118 |
This checkpoint: Arena-Hard 0.6111 (95% prompt bootstrap [0.5823, 0.6394], 495 prompts); AlpacaEval 0.4118 (95% prompt bootstrap [0.3944, 0.4304], 805 prompts).
Two limits these numbers carry
- They are not leaderboard scores. Arena-Hard normally judges with
gpt-4-1106and AlpacaEval 2.0 withweighted_alpaca_eval_gpt4_turbo. To keep paid-API cost at zero, judging here uses a local open-weight model. The questions and baseline answers are the official static files, so the three arms are comparable with each other, but the absolute values are not comparable with published leaderboard numbers. Re-judging the same responses with a larger open-weight judge moved every arm up by about 0.11 and changed the ranking, which is why only within-judge comparisons are reported. - The differences between the trained arms are not statistically resolved. Paired whole-prompt bootstraps on the common prompts give NBPO minus PROSPER +0.0087 [-0.0122, +0.0305] on Arena-Hard and +0.0063 [-0.0087, +0.0206] on AlpacaEval. NBPO minus fixed-reference Nash does exclude zero on Arena-Hard (+0.0243 [+0.0040, +0.0446]), but that arm also fell -0.0250 below its own single-epoch run, so the gap reflects the fixed reference degrading rather than NBPO improving. General-capability benchmarks (MMLU, ARC-Challenge, HellaSwag, IFEval) separate none of the arms from each other or from the base by more than their standard errors.
The three arms
promotion/NBPO-Qwen2.5-7B-WildChecklists-2eppromotion/PROSPER-Qwen2.5-7B-WildChecklists-2eppromotion/FixedRefNash-Qwen2.5-7B-WildChecklists-2ep
The tokenizer is unchanged from the base model.