Qwen3.8-27B-DFlash2 for the tiyuvta serving engine
The DFlash 2 block-diffusion draft model for
Qwen/Qwen3.8-27B, packaged for serving on the
tiyuvta serving engine. The weights are byte-identical to
z-lab/Qwen3.8-27B-DFlash2
(model.safetensors sha256 67fc76d68dc5a9415511a4f394ef744d67510cd20e93b37cc2cc7d28e4bab65c),
itself a mirror of incoai/Qwen3.8-27B-DFlash2. Apache-2.0, unmodified; this card adds the
serving configuration and measured receipts.
It is not a standalone language model: it drafts token blocks that the target model verifies. Greedy output is byte-identical to plain decode by construction — the verifier arbitrates every committed token.
Serving
Served on the tiyuvta serving engine, tuned for this model on RTX PRO 6000 Blackwell. The
serving configuration: the OpenAI-compatible API in front of the
tiyuvta/Qwen3.8-27B-NVFP4-MTP-GGUF
trunk (the Q5K-mtp build), the DFlash2 speculative route armed with this repository as
the drafter, and the FR-Spec ranks file loaded.
Defaults do the tuning (serving engine >= v0.112.0): the drafter backbone quantizes to
q4_0 at load (the measured default), and with the ranks file armed the round consumes an
FR-Spec vocabulary trim — the draft-side lm_head and its top-k run over 32,768 ranked rows
instead of the full 248,320 while verification keeps the full vocabulary, so output is
still byte-identical to plain decode. The ranks file is
q38-ranks-sxc32768.gguf.txt.
Boot receipt: [dspark] q38: DFlash2 draft head TRIMMED to 32768 rows.
Arming the DFlash2 route gives this drafter the model's speculative program (the MTP arm
disengages; two spec programs never coexist). Exactness gate: spec-on vs spec-off streams are
byte-identical (dspark_q38_gate, ALL EXACT).
Measured (RTX PRO 6000 Blackwell, served, greedy, conc 1 steady, x3 interleaved)
| config | chat agg tok/s | agentic agg tok/s | accept |
|---|---|---|---|
| native MTP masked head (baseline) | 126.9 | 148.6 | 0.63 |
| DFlash2, q8 drafter, full vocab | 134.9 | 151.5 | 0.75 |
| DFlash2, q4 drafter, full vocab | 136.3 | 154.5 | 0.76 |
| DFlash2, q4 drafter + FR-Spec trim (defaults) | 142.9 | 157.2 | 0.73 |
| DFlash2, fixed verify T=8 (rejected) | 132.0 | 145.5 | 0.52 |
- Adaptive verify width (the default) beats fixed T=8 at serve shape.
- The q4 drafter re-encode costs nothing in acceptance and saves draft time; the FR-Spec vocab trim stacks on it (bare gate: 152.4 -> 160.4 tok/s combined, +5.2%). Both ship as defaults in serving engine v0.112.0, alongside the concurrency shed that keeps speculation low-concurrency-only under load (c>=4 aggregate at plain-batch parity).
- The trim trades a little acceptance (0.755 -> 0.730 agentic) for the wall win — chat gains most (+4.8% served) because shorter rounds make the head cost a bigger share.
- The asymmetric q5 quant collapses acceptance (0.66 -> 0.42) and is not offered.
Served (serving engine >= v0.113.0): sampled admission stacks, and the route carries production
The measured tables above are GREEDY — the byte-exactness instrument. Production serves the vendor-recommended SAMPLED defaults, and since v0.113.0 sampled DFlash2 sessions stack in the spec-gate LOW band like greedy ones (earlier versions admitted sampled speculation only on an otherwise-idle server). At that shape, on the same RTX PRO 6000 (x3 interleaved, medians, agg tok/s vs the MTP head):
| c | DFlash2 route | MTP head |
|---|---|---|
| 1 | 126.8 | 116.6 |
| 2 | 127.5 | 119.5 |
| 4 | 87.1 | 84.8 |
Single-stream wall rates on the same card scale with output predictability: prose ~131-146, code ~208-239, digit-heavy ~287-339 tok/s. This configuration serves both production Qwen3.8 origins of the maintainers' own API.
Multi-turn (serving engine >= v0.112.0)
Conversations resume their parked session by default: turn N+1 primes only the new suffix
onto the parked trunk cache + draft KV instead of re-prefilling the whole conversation.
Measured on the same card, 8-turn conversation: per-turn TTFT -16% -> -83% vs cold
re-prime, growing with depth (turn 7: 0.32 s vs 1.90 s). Resume receipt:
[worker] dspark-reuse: N committed tokens resumed. Streams cut by max_tokens mid-burst
do not park, and a request whose context cap outgrew the parked allocation serves cold —
both named in the log; agent loops that omit max_tokens resume every turn.
Citations
@misc{inco2026dflash2,
title = {{DFlash 2: Keep Drafting Parallel}},
author = {{Inco AI}},
year = {2026},
month = {August},
url = {https://inco.ai/blog/dflash2/}
}
@inproceedings{chen2026dflash,
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
booktitle = {International Conference on Machine Learning (ICML)},
year = {2026}
}