Olmo3-B3PO-7B

📄 Paper: Parameter Exploration for RLVR via Variational Learning · arXiv:2608.09805

📦 Code: insait-institute/c3po

Olmo-3 7B fine-tuned with B3PO (Bayesian Perturbed Parameter Policy Optimization), from the paper "Parameter Exploration for RLVR via Variational Learning".

3PO is a family of parameter-space exploration strategies for Reinforcement Learning with Verifiable Rewards (RLVR). Instead of relying only on action-space heuristics (temperature, clipping, entropy bonuses), 3PO samples model weights from an approximate posterior learned with the variational optimizer IVON; the amount of weight noise becomes an extra control lever for exploration.

B3PO draws a single weight perturbation from the IVON posterior per gradient step and syncs it to the rollout engine, so each step explores from one perturbed parameter sample.

Training

Evaluation

Evaluated on AIME 2024–2026, MATH-500, AMC 2023, and Minerva. See the paper for full results.

Usage

Loads as a standard causal LM:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("BayesRL/Olmo3-B3PO-7B")
tok = AutoTokenizer.from_pretrained("BayesRL/Olmo3-B3PO-7B")

Citation

@misc{venkatkrishna2026parameterexploration,
      title={Parameter Exploration for RLVR via Variational Learning}, 
      author={Vatsal Venkatkrishna and Nico Daheim and Iryna Gurevych},
      year={2026},
      eprint={2608.09805},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2608.09805}, 
}