EAGLE-3 draft head for Qwen3-0.6B (tool-calling tuned)
A speculative-decoding draft head trained with SpecForge
for Qwen/Qwen3-0.6B, aimed at fast single-user JSON tool calling. Qwen ships MTP/NEXTN heads only
for its larger models, so nothing existed for this size: this head was trained in-house.
Current revision = v3, trained in three stages on an 85k-conversation mix (40k
glaive-function-calling-v2 + 30k ultrachat + 15k synthetic water/IoT tool calls whose argument values
are BOUNDED: enums, structured ids, fixed units, which is what production tool schemas look like).
The last stage adds 42k steps at ttt_length 7 and max_length 4096, the two settings the earlier
revisions had to give up on a shared GPU.
Measured (H100 PCIe, quiet card, SGLang 2026-07-20 nightly, fa3, 500-token decode)
| Config | Single-stream |
|---|---|
| Qwen3-0.6B, no speculation | 553 tok/s |
| + this head, v1 (40k steps) | 814 tok/s |
| + this head, v2 | 861 tok/s |
| + this head, v3 (current) | 955 tok/s (+73%) |
By generation domain, v3 versus v2 under identical conditions: generic 924 vs 650, free-form JSON 782 vs 566 (+38%), bounded-argument domain JSON 757 vs 592 (+28%). The domain dataset pays where argument values are enums and ids rather than free text, which is the production case.
Speculation config sweep: steps 3 / topk 1 / draft 4 is the optimum. Deeper or wider LOSES
(steps 4: 705, steps 5: 631, topk 2: 757 on v1): verification cost outgrows acceptance on a 0.6B
target. ngram speculation is a net regression at batch 1 (426-445 versus 665 baseline).
Usage (SGLang)
python -m sglang.launch_server \
--model-path Qwen/Qwen3-0.6B \
--speculative-algo EAGLE3 \
--speculative-draft-model-path Nicolassuez/Qwen3-0.6B-eagle3 \
--speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
--attention-backend fa3
Serving footprint measured at ~3.5 GiB (context 4096).
Training notes
- Draft: 1-layer llama-style EAGLE-3 (hidden 1024, draft vocab 32k), auxiliary hidden states captured from target layers 1/13/24.
- ttt_length 7, max_length 4096, batch size 1, lr 1e-4. Earlier revisions used ttt 5 / 2048 because the training GPU was shared with production services; lifting both is what took 861 to 955.
- Trained in completable chunks, each warm-started from the previous one.
- The vocabulary mapping (d2t/t2d) is baked in; any continuation must reuse the same mapping.