SUPERSEDED: NBPO on UltraFeedback (contaminated training split)
Do not use this checkpoint for the UltraFeedback results in the NBPO paper. It is kept only so that the published artifact remains inspectable.
What is wrong with it
An audit by prompt-text hash found that 629 of the 698 UltraFeedback evaluation prompts appear in this checkpoint's training pairs. The panel's response pool was regenerated for this campaign with prompts drawn independently of the earlier campaign from which the evaluation list was inherited, and the two collided. Every other UltraFeedback arm in the paper is prompt-disjoint from the evaluation set, so this checkpoint was the only one measured partly in sample, and the bias runs in its own favour.
Replacement
promotion/Llama-3.1-8B-NBPO-UltraFeedback-clean is the same configuration retrained on pairs
filtered to be prompt-disjoint from the evaluation set (37,800 -> 34,026 training rows over 5,671
prompts). It is the checkpoint behind the paper's UltraFeedback NBPO row.