Open-MOPD-SmolLM3-3B-RL-IF
This is the instruction-following (IF) domain teacher in the Open-MOPD
pipeline. It starts from BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT and
is trained only on instruction-following prompts with verifiable rewards using
GRPO. This release corresponds to training step 2,440.
Because instruction-following responses are shorter, training uses a
2,048-token response limit, rollout group size 8, and 3,000 planned steps.
Other settings match the math and code teachers: global batch size 128,
mini-batch size 32, learning rate 1e-6, no KL penalty, and accuracy-based
group filtering.
Results
| Model | IFEval | IFBench_test | IF average |
|---|---|---|---|
| RL-IF teacher | 74.49 | 27.67 | 51.08 |
| MixSFT starting point | 66.91 | 16.00 | 41.46 |
Results use n=1, temperature 1.0, enable_thinking=true,
max_model_len=32768, top_p=0.95, top_k=-1, and
stop_token_ids=[128012].
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-IF"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
Intended use and limitations
This is a domain teacher intended for distillation, not a general-purpose assistant. It was optimized only for instruction following and can perform worse than MixSFT on other domains.
Instruction following has the largest teacher gain in this pipeline (+9.62 points over MixSFT). It is also the domain most harmed by naive multi-teacher OPD: without token-budget balancing, it receives only about 1% of weighted gradient tokens.
Model specifications
- Architecture:
SmolLM3ForCausalLM - Parameters: approximately 3B
- Layers: 36
- Vocabulary size: 128,256
- Weights: BF16, approximately 6.2 GB
- Includes tokenizer and chat template