math_think_s1
Full-parameter SFT of Qwen3-4B-Base for math reasoning with explicit thinking blocks (qwen3 template + enable_thinking: true).
Base model
Training data
- Dataset:
math_think_s1— 75,905 ShareGPT samples - Source: qihoo360/Light-R1-SFTData (
stage1-76k.json) - Decontamination: MinHash + numeric normalization vs AIME/MATH eval prompts (1 sample removed)
Training recipe
| Item | Value |
|---|---|
| Framework | LLaMA-Factory |
| Method | Full SFT, DeepSpeed ZeRO-3 |
| Template | qwen3 (reasoning / thinking) |
| Cutoff | 32768 |
| Packing | true |
| Epochs | 4 |
| LR | 1e-5 (cosine, warmup 10%) |
| Batch | 1 × 16 grad accum × 4 GPUs = 64 |
| Steps | 944 |
| Train loss | 0.6066 |
| Runtime | ~39.3 h |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "YOUR_HF_USERNAME/math-think-s1-qwen3-4b" # replace after upload
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Find the sum of all positive integers n such that n^2 + 12n - 2007 is a perfect square."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
Files
Release tarball includes: model.safetensors, config.json, generation_config.json, tokenizer files, chat_template.jinja, README.md, train_results.json, trainer_log.jsonl, MANIFEST.txt.