Qwen2.5-Math-7B-ES-MATH
Qwen/Qwen2.5-Math-7B post-trained with Evolution Strategies (ES) for mathematical reasoning, from
Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies.
ES is a population-based, gradient-free method that optimises directly in weight space through random perturbations. Unlike RL, which sharpens the output distribution around its single best guess, ES raises pass@1 while preserving solution coverage — so the model keeps benefiting from additional test-time samples. See the paper repository.
Training setting
This model was trained in the RL-Zero setting: post-training applied directly to the base model with outcome-verified rewards, with no supervised fine-tuning or distillation beforehand.
- Data — the MATH dataset restricted to levels 3–5 (Hendrycks et al., 2021; Liu et al., 2025)
- Reward — binary, verifier-checked correctness of the final
\boxed{}answer - Maximum response length during training — 3,000 tokens
Training parameters
As reported in the paper appendix (Table 2):
| Hyperparameter | Value |
|---|---|
| Perturbation scale σ | 0.001 |
| Learning rate α | 0.0005 (σ / 2) |
| Population size | 32 |
| Train batch size | 512 |
| Max response length | 3,000 |
| Rollout temperature | 0.0 |
| Total training steps | 500 |
Evaluation
Following Yue et al. (2025), all benchmarks are sampled at:
| Parameter | Value |
|---|---|
| Temperature | 0.6 |
| Top-p | 0.95 |
| Max tokens | 16,384 |
Reproduce with the evaluation harness:
python -m es_pass_k.math_pass_k \
--model-path conorfhayes/Qwen2.5-Math-7B-ES-MATH \
--eval-dataset datasets/evaluation_suite/math/math500 \
--k 256 --temp 0.6 --top-p 0.95 --max-tokens 16384 \
--template math \
--use-gpus 0
Usage
The model expects the Qwen2.5-Math prompt format, with the reasoning instruction in the system turn. Using a different template is out-of-distribution and will depress accuracy:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("conorfhayes/Qwen2.5-Math-7B-ES-MATH", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("conorfhayes/Qwen2.5-Math-7B-ES-MATH")
question = "What is the smallest positive integer n such that n! is divisible by 1000?"
prompt = (
"<|im_start|>system\nPlease reason step by step, and put your final answer "
"within \\boxed{}.<|im_end|>\n<|im_start|>user\n"
+ question
+ "<|im_end|>\n<|im_start|>assistant\n"
)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
max_new_tokens=2048, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0], skip_special_tokens=True))
Sampling matters: this model is intended for repeated sampling, where its broader output distribution is the advantage. Greedy decoding discards exactly the property ES optimises for.
Limitations
- Trained and evaluated only on English mathematical reasoning. Behaviour on other domains, languages, or task formats is uncharacterised.
- Inherits the capabilities and biases of
Qwen/Qwen2.5-Math-7B. - No safety alignment was performed. This is a research artefact for studying solution coverage under test-time scaling, not a deployment-ready assistant.
- Reward was outcome-based only, so correct answers reached by faulty reasoning were rewarded.
Citation
@article{hayes2026beyond,
title = {Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies},
author = {Hayes, Conor F. and Meyerson, Elliot and Schweighofer, Kajetan and
Dailey, Roberto and Hodjat, Babak and Miikkulainen, Risto and Qiu, Xin},
year = {2026}
}