Qwen3-8B-Base -- OpenThoughts3 math SFT, step 750
Intermediate checkpoint (step 750 of a planned 2000), bf16 weights only. Not yet evaluated.
| base | Qwen/Qwen3-8B-Base |
| data | knowledge-distillation/openthoughts3_math, 103,760 two-turn conversations |
| sequence length | 32768 (truncates nothing: mean 13.8k, p99 16.8k, max 16.8k tokens) |
| global batch | 255 |
| optimizer | AdamW, lr 5e-6, cosine, 3% warmup, weight decay 0.01, grad clip 1.0 |
| precision | bf16, FSDP over 3 GPUs, verl |
Every assistant message in the data is a <think>...</think> trace ending in \boxed{}, and the
model is trained to reproduce that. It uses a plain ChatML template with no thinking logic --
Qwen3's own template strips <think> from all but the last assistant turn, which would have
removed the reasoning from the training targets. No enable_thinking flag applies.
A turn ends with <|im_end|> (151645); config.json and generation_config.json both say so.
Qwen3-8B-Base's stock generation_config.json says 151643 instead, which yields a model that
never terminates under vLLM. Generate with max_new_tokens around 16384.