NBPO (per-prompt inverse-surplus weights)
Research checkpoint. Nash bargaining over game-value improvements with an adaptive (self-play) reference, with the dual multipliers fitted separately at each prompt.
Fine-tuned from Qwen/Qwen2.5-7B-Instruct. This repository is one of three arms trained and evaluated
under one protocol so that the only difference between them is the
aggregation rule; the other two are linked below.
Training setting
Retrained under the protocol of "Back to Blackwell" (PROSPER) on WildChecklists, where every prompt carries its own checklist items and the judge scores one item at a time.
| Prompts | 522 (union-filtered so all three arms train on one prompt set) |
| Learner pairs | 14,616 (all 28 unordered pairs of 8 sampled responses per prompt) |
| Pipeline epochs | 2 |
| Optimizer steps | 228 at batch size 128 |
| Learning rate | 3e-7, weight decay 1e-6, AdamW, warmup ratio 0.1, grad clip 1.0 |
| Max sequence / prompt length | 2048 / 1024 |
| Seed | 555134 |
| Candidate decoding | temperature 0.8, top_p 0.9, 8 responses per prompt |
| Judge for training signal | local open-weight judge, 10 judgments per pair averaged over both presentation orders |
Provenance recorded by the run: dataset manifest 8d0dc02928c1643b, solver artifact not recorded.
Evaluation
Greedy generation (temperature 0) against each benchmark's released baseline answers, judged by a local open-weight judge at temperature 0 in both presentation orders with ties counted as 0.5.
| Arm | Arena-Hard | AlpacaEval |
|---|---|---|
| NBPO | 0.6333 | 0.4124 |
| PROSPER | 0.6225 | 0.4071 |
| Fixed-reference Nash | 0.6111 | 0.4118 |
This checkpoint: Arena-Hard 0.6333 (95% prompt bootstrap [0.6045, 0.6616], 495 prompts); AlpacaEval 0.4124 (95% prompt bootstrap [0.3946, 0.4311], 802 prompts).
Two limits these numbers carry
- They are not leaderboard scores. Arena-Hard normally judges with
gpt-4-1106and AlpacaEval 2.0 withweighted_alpaca_eval_gpt4_turbo. To keep paid-API cost at zero, judging here uses a local open-weight model. The questions and baseline answers are the official static files, so the three arms are comparable with each other, but the absolute values are not comparable with published leaderboard numbers. Re-judging the same responses with a larger open-weight judge moved every arm up by about 0.11 and changed the ranking, which is why only within-judge comparisons are reported. - The differences between the trained arms are not statistically resolved. Paired whole-prompt bootstraps on the common prompts give NBPO minus PROSPER +0.0087 [-0.0122, +0.0305] on Arena-Hard and +0.0063 [-0.0087, +0.0206] on AlpacaEval. NBPO minus fixed-reference Nash does exclude zero on Arena-Hard (+0.0243 [+0.0040, +0.0446]), but that arm also fell -0.0250 below its own single-epoch run, so the gap reflects the fixed reference degrading rather than NBPO improving. General-capability benchmarks (MMLU, ARC-Challenge, HellaSwag, IFEval) separate none of the arms from each other or from the base by more than their standard errors.
The three arms
promotion/NBPO-Qwen2.5-7B-WildChecklists-2eppromotion/PROSPER-Qwen2.5-7B-WildChecklists-2eppromotion/FixedRefNash-Qwen2.5-7B-WildChecklists-2ep
The tokenizer is unchanged from the base model.