Panacea on TL;DR
Panacea (Zhong et al., NeurIPS 2024) with the DPO procedure and linear scalarisation: SVD-LoRA with k=8 preference-agnostic singular values and the preference vector injected as the remaining singular values, merged at the operating point chosen on validation prompts only.
- Panel: TL;DR
- Objectives: coverage, faithfulness, conciseness, helpfulness
- Backbone / reference policy:
meta-llama/Llama-3.1-8B-Instruct - Training budget: 300 optimiser updates, global batch 16
- Reported in: Nash Bargaining Preference Optimization (NBPO), Table 2 (primary cross-method evaluation)
Evaluation protocol: independent objective-wise win rate against the common reference,
judged by Llama-3.3-70B-Instruct on prompt-disjoint held-out prompts, both presentation orders.