OPDLM-MATH-4B

OPDLM-MATH-4B is an On-Policy Distillation Language Model (OPDLM) — a block-diffusion language model (block size 4, 4 denoising steps per block) post-trained for mathematical reasoning. It is built on a block-diffusion adaptation of Qwen/Qwen3-4B (architecture a2d-qwen3).

This is the base (non-thinking) variant: short-context (2k) math post-training, no explicit chain-of-thought thinking block. ArXiv report: arxiv.org/abs/2606.06712

arXiv report: arxiv.org/abs/2606.06712

Usage

This model uses custom modeling code; load with trust_remote_code=True. Generation is block-diffusion (non–left-to-right), so use the project's inference utilities (block_size=4, denoising_steps_per_block=4) rather than vanilla model.generate.

from transformers import AutoModel, AutoTokenizer
tok = AutoTokenizer.from_pretrained("divelab/OPDLM-MATH-4B", trust_remote_code=True)
model = AutoModel.from_pretrained("divelab/OPDLM-MATH-4B", trust_remote_code=True)