Qwen3-8B, speculative reasoning — 1 epoch
Qwen3-8B fine-tuned to hand work off mid-reasoning and keep going while it is checked. The model writes
<Pending>the work, stated as a task</Pending>
<Predict>what it expects that work to conclude</Predict>
and then stops. A runtime carries the task out in a separate branch while the main line
continues under <Assumed/>. If the check agrees, the continuation stands. If it does not,
the branch is spliced back in as <Resolve>…</Resolve><Rollback/> and the model resumes
from the real result.
Training
- data: polaris-easy2k-qwen3-8b-speculative-sft,
sft/units-overhead1.0.jsonl— 18,863 units from 1,903 traces, 76.9M tokens - 1 epoch, lr 1e-5 cosine to zero, warmup 0.05, weight decay 1e-4, Adam β 0.9/0.95, effective batch 16 sequences, bf16, ZeRO-3, 8×H100, 42 minutes
- loss only on the completion; the handed-in prefix is masked
- the twelve tags are added as special tokens, initialised from the mean embedding of their
own words.
<|im_end|>is trained only where generation genuinely ends — never after</Predict>, which is a hand-off, not a stopping point.
Code: yurun-yuan/speculative-reasoning
(threadweaver/threadweaver_sft/src/sft_speculative.py, docs/training.md).
Evaluation
RyanYr/MathEval, 1,547 problems, one sample at temperature 0.6 / top-p 0.95 / top-k 20, 20,000 token budget.
| Qwen3-8B | this model | |
|---|---|---|
| MathEval overall | 68.78% | 66.97% |
| math500 | 92.6% | 90.4% |
| olympiadbench | 61.8% | 59.7% |
| minerva | 42.3% | 43.8% |
| amc23 | 82.5% | 87.5% |
| aime24 | 70.0% | 46.7% |
| aime25 | 50.0% | 43.3% |
Paired over the same problems: 989 both right, 75 base only, 47 this model only.
The structure is produced: 1,288 of 1,547 problems hand off at least once, 1,973 sites in total, of which 1,489 MATCH and 484 MISMATCH.
It does not yet save time
| tokens per problem | |
|---|---|
| base, all sequential | 8,057 |
| this model, spine only | 7,877 |
| this model, critical path | 8,240 |
| this model, total generated | 8,713 |
At a matched site the check overlaps the continuation and costs nothing; at a rolled-back site the continuation is discarded and the check lands on the critical path. Split that way: −7.5% on the 844 problems where every site matched, +11.9% on the 444 where something rolled back, +2.3% overall.
Two gaps against the training data explain it. The model hands off at 0.16 sites per 1k tokens against 0.55 in the units, and 24.5% of its predictions are wrong against 7.7% in the units. The training sites were chosen by reading a finished trace, with hindsight; the model has to make the same call prospectively.
Generating with it
</Predict> must be a stop string, and the tags are special tokens — with vLLM's default
skip_special_tokens=True the detokenizer deletes them and the stop string can never match.
Set skip_special_tokens=False. A working runtime is in
threadweaver/threadweaver_sft/src/eval_matheval.py (--mode speculative).
Also here
checkpoint-1000/ is a mid-training snapshot (step 1,000 of 1,179), kept for the loss
trajectory. The model at the repository root is the finished one.