Open-MOPD-SmolLM3-3B-Final
This is the final flagship model from the Open-MOPD pipeline. Starting from
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT, it is trained with multi-teacher
online policy distillation using three domain-specific RL teachers for math,
code, and instruction following. This release corresponds to training step 200.
Each prompt is hard-routed to its domain teacher. The dense reward is the
teacher-student log-probability gap over the student's top-k distribution
(k=16, nucleus p=0.99), weighted by the student probabilities and applied
directly as the token-level advantage.
Open-MOPD uses three mechanisms to prevent cross-domain training imbalance:
- Token-share balancing keeps the weighted gradient-token share near one third per domain. Without it, math and code consume about 99% of gradient tokens.
- Gap-aware allocation (
alpha=1) shifts the training budget toward domains with more remaining teacher-student improvement. - Reward refresh recomputes the student-dependent part of the dense reward at every inner update without adding extra prefills.
Training uses a global batch size of 1,024, mini-batch size 256, constant
learning rate 1.5e-6, clipping at 0.2/0.28, no KL penalty, and a domain
sampling ratio of math:code:IF = 2:2:1.
Results
| Method | Math | Code | IF | Overall | Recovery |
|---|---|---|---|---|---|
| RouteRL teacher upper bound | 24.24 | 21.73 | 51.08 | 32.35 | 100% |
| Open-MOPD Final | 22.42 | 21.73 | 49.58 | 31.24 | 83.4% |
| Naive multi-teacher OPD | 21.26 | 19.26 | 43.64 | 28.05 | 35.6% |
| MixSFT starting point | 17.95 | 17.60 | 41.46 | 25.67 | - |
Per-dataset scores are AIME24 21.98, AIME25 22.86, LiveCodeBench v5 20.84, LiveCodeBench v6 22.63, IFEval 74.49, and IFBench_test 24.67. Recovery measures the fraction of the overall score gap between MixSFT and the routed-teacher upper bound recovered by the final student.
Evaluation protocol
Results are averaged per dataset, then per domain, followed by a macro-average over the three domains.
- Math: AIME24 and AIME25, avg@64, temperature 0.6.
- Code: LiveCodeBench v5 and v6, avg@10, temperature 1.0.
- Instruction following: IFEval and IFBench_test,
n=1, temperature 1.0, withenable_thinking=true.
All evaluations use max_model_len=32768, top_p=0.95, top_k=-1, and
stop_token_ids=[128012]. Code is reported as avg@10, not best@10.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-Final"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype="bfloat16",
device_map="auto",
)
messages = [{"role": "user", "content": "Find all real solutions of x^3 - 3x + 1 = 0."}]
inputs = tokenizer.apply_chat_template(
messages,
return_tensors="pt",
add_generation_prompt=True,
enable_thinking=True,
).to(model.device)
outputs = model.generate(
inputs,
max_new_tokens=8192,
temperature=0.6,
top_p=0.95,
do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
Single greedy generations are not comparable to the reported benchmark results; use the sampling protocol above for reproduction.
Model family
SmolLM3-3B-Base -> MixSFT initialization -> Math/Code/IF RL teachers ->
Open-MOPD Final.
The full family and training/evaluation data are released under the
BytedTsinghua-SIA organization.
Model specifications
- Architecture:
SmolLM3ForCausalLM - Parameters: approximately 3B
- Layers: 36
- Vocabulary size: 128,256
- Weights: BF16, approximately 6.2 GB
- Context used in evaluation: 32,768 tokens
- Includes tokenizer and chat template