SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering

This repository contains the fine-tuned Llama-3.1-8B model checkpoint developed using the SPADER reinforcement learning framework, as presented in the paper SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering.

Model Description

SPADER is a reinforcement learning framework designed for long-horizon tool-use agents in Multi-Answer QA. It introduces:

• Step-wise Peer Advantage (SPA): A critic-free step-level credit assignment mechanism that aligns parallel trajectories by decision step and estimates advantages from peer returns.
• Diversity-Aware Exploration Reward: Promotes long-tail entity discovery by upweighting rare findings and downweighting redundant ones.

This checkpoint represents the Llama-3.1-8B-Instruct base model trained with SPADER.

• Repository: KhanCold/spader
• Paper: arXiv:2606.00593

Citation

@misc{shi2026spaderstepwisepeeradvantage,
      title={SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering}, 
      author={Qiming Shi and Zhaolu Kang and Yunfan Zhou and Di Weng and Yingcai Wu},
      year={2026},
      eprint={2606.00593},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.00593}, 
}