Qwen3.8-27B-DFlash2-b32
A DFlash 2 draft model for Qwen/Qwen3.8-27B, fine-tuned from z-lab/Qwen3.8-27B-DFlash2 for longer drafts. It is not a standalone language model: it proposes tokens that the target model then verifies (speculative decoding). Made for qwen-spec, which runs Qwen3.8-27B on Apple silicon and uses this head by default (write-up).
What changed
A DFlash 2 head guesses a whole block of upcoming tokens in one pass. The original head was trained for blocks of 8 tokens, and its guesses beyond the 15th position are rarely right. This head was trained for blocks of 32 tokens on Qwen3.8's own thinking-mode answers: positions 1 to 8 are as accurate as before, positions 9 to 31 are right about 25% more often. The gain shows up when the drafts are long enough to use those positions.
Accepted tokens per pass on a MacBook Pro (M5, 4-bit Qwen3.8-27B, exact sampling), each head with a draft tree of 32 tokens:
| original head | this head | |
|---|---|---|
| GSM8K (thinking) | 6.0 | 7.0 (+16%) |
| chat (thinking) | 3.9 | 4.2 (+7%) |
| HumanEval | 11.9 | 15.5 (+30%) |
Training
- Data: 25.7K prompts from public chat, math and code sets, each answered once by Qwen3.8-27B in thinking mode with its recommended sampling settings (110M tokens). The permissively licensed part (17K prompts, 54M tokens) is published as JonasLoos/Qwen3.8-27B-thinking-completions.
- Target features: hidden states of the 4-bit MLX checkpoint (lmstudio-community/Qwen3.8-27B-MLX-4bit).
- Objective: the DFlash 2 training objective from SpecForge at block size 32.
- Optimizer: AdamW, learning rate 1e-5 with cosine decay and 4% warmup, 2 epochs, about 15 hours on one H100.
Usage
uv tool install git+https://github.com/JonasLoos/qwen-spec
qwen-spec "your prompt"
SGLang and vLLM, which serve the original head, should accept this one as well; only qwen-spec's MLX engine has been tested.