NBPO on UltraFeedback (contamination-free rebuild)
The UltraFeedback NBPO arm retrained on pairs filtered to be prompt-disjoint from the 698 evaluation prompts (37,800 -> 34,026 rows over 5,671 prompts). Identical configuration to the released arm otherwise: eta_t = 1, lr 5e-7, 300 updates, batch 2 x 8, Adafactor.
- Panel: UltraFeedback
- Objectives: instruction following, truthfulness, honesty, helpfulness
- Backbone / reference policy:
meta-llama/Llama-3.1-8B-Instruct - Training budget: 300 optimiser updates, global batch 16
- Reported in: Nash Bargaining Preference Optimization (NBPO), Table 2 (primary cross-method evaluation)
Evaluation protocol: independent objective-wise win rate against the common reference,
judged by Llama-3.3-70B-Instruct on prompt-disjoint held-out prompts, both presentation orders.