Qwen3.8-27B-DFlash2 — imatrix-calibrated GGUF drafts
Two quantizations of the DFlash 2 draft model for
Qwen/Qwen3.8-27B, both calibrated with an
importance matrix captured from the drafter's own activations during live speculative decoding.
This is not a standalone language model. It runs inside a speculative-decoding server and drafts tokens for a target model to verify. On its own it produces nothing useful.
| File | Size | Acceptance | Use when |
|---|---|---|---|
Qwen3.8-27B-DFlash2-Q3_K_M.gguf | 916,703,008 | 65.55% | you have the memory — matches the 1.14 GB Q4_K_M |
Qwen3.8-27B-DFlash2-Q2_K.gguf | 705,431,328 | 64.08% | memory is tight — 211 MB less for 1.5 points |
Why these exist
The upstream repositories publish BF16, Q8_0 and Q4_K_M, none calibrated. Calibrating a DFlash 2
drafter is not straightforward: llama-imatrix cannot run one at all, because the architecture
has no output layer and no token embedding and borrows both from its target, so it cannot form a
context alone (dflash requires ctx_other to be set). The collector has to be attached to the
draft context inside a running speculative session instead.
Two filters in llama.cpp's collector then discard most of what matters. It skips any batch with
fewer than 16 rows — and DFlash 2's draft step evaluates only n_max + 1 tokens, so every
draft-decode matmul is rejected — and it only collects tensors named blk.*, which excludes
DFlash 2's own top-level fc.weight and selector_hidden.weight. Attaching the stock collector
captures 10 of 49 tensors. Relaxing both filters captures 49 of 49.
That difference is the whole point of these files. A patch is included as
dflash-imatrix.patch so the calibration is reproducible.
Measurements
One machine, one prompt set, one target, all arms identical apart from the drafter: AMD RX 7800
XT, Vulkan, llama.cpp 1deefcca, a quantized Qwen3.8-27B target, --spec-draft-n-max 4,
16k context, 20 fixed prompts, temperature 0, seed 42, 256-token cap, one slot.
| Drafter | Bytes | Micro acceptance |
|---|---|---|
| Q4_K_M (upstream, uncalibrated) | 1,143,006,752 | 65.56% |
| Q3_K_M (this repo, calibrated) | 916,703,008 | 65.55% |
| Q3_K_M (uncalibrated control) | 916,702,752 | 65.20% |
| Q2_K (this repo, calibrated) | 705,431,328 | 64.08% |
| Q2_K (uncalibrated control) | 705,431,072 | 61.11% |
The calibrated Q3 matches the upstream Q4_K_M — 65.55% against 65.56% — at 226 MB less. Calibration is worth +0.35 points at 3 bits and +2.97 points at 2 bits, which is where it really pays: an uncalibrated Q2_K gives up 4.45 points against Q3, a calibrated one only 1.47.
Acceptance is deterministic in this stack and reproduces exactly on repeat runs, which is why it is the only figure reported here. Tokens per second is not, because for a target this size it is governed by how much of your setup fits in VRAM rather than by the drafter — measure it on your own machine, at your own context length.
If you need something smaller still
HermiHg/Qwen3.8-27B-DFlash2-Q2_K_S-MIX-GGUF
is a mixed 2–3-bit build at 561 MB — another 144 MB below the Q2_K here. Measured under the same
protocol on the same target it reaches 62.50%, so it trades 1.58 points of acceptance for
that space. If you are tight enough on memory that 144 MB matters, it is the better choice, and
it comfortably beats an uncalibrated Q2_K at any size.
Provenance
Quantized from Qwen3.8-27B-DFlash2-BF16.gguf (3,860,293,152 bytes) as published in
z-lab/Qwen3.8-27B-DFlash2-GGUF,
itself a conversion of
incoai/Qwen3.8-27B-DFlash2.
llama.cpp revision 1deefcca395743049c3820ab8f9b15043f3e9446 plus dflash-imatrix.patch.
The imatrix was captured from a live speculative session against a quantized Qwen3.8-27B target,
with both collector filters relaxed so draft-decode batches and DFlash 2's top-level weights are
included.
Pipeline control. Rebuilding Q4_K_M from the same BF16 with an unpatched build yields a file byte-identical to the published one:
18a380efc9b7ed8d88677fc895f5c11ae170653434ee378f7348f715c14d0594
matching z-lab's Qwen3.8-27B-DFlash2-Q4_K_M.gguf exactly. These builds therefore come from a
pipeline demonstrably able to reproduce the upstream artifacts.
Usage
llama-server -m <your-Qwen3.8-27B-target>.gguf \
--spec-type draft-dflash \
-md Qwen3.8-27B-DFlash2-Q3_K_M.gguf \
--spec-draft-n-max 4 \
--spec-draft-ngl 99
n=4 and n=5 measured as statistically indistinguishable; n=6 costs acceptance and gains
nothing for it.
Requires a llama.cpp build with DFlash 2 support. The patch is only needed to reproduce the calibration — the GGUFs themselves load on any DFlash 2 build.
Note DFlash 2 carries a few GB of overhead beyond the drafter's own weights, from hidden-state taps and verification buffers, which is comfortable on a large-memory machine and can be prohibitive on a 16 GB GPU.
Credit
DFlash 2 is the work of the z-lab / inco.ai authors (blog, code). This repository redistributes re-quantized copies of their released drafter and claims no part of the method, the training, or the original model. Apache-2.0, inherited from upstream.