GLM-5.3-Flash DFlash2 Drafter — run dflash2G

Self-trained DFlash2 block-diffusion speculative-decoding drafter for GLM-5.3-Flash, trained against our INT4 quant canada-quant/GLM-5.3-Flash-W4A16-MTP. Trained on self-generated data only — no third-party drafter weights or traces anywhere in the training path. Supersedes GLM-5.3-Flash-DFlash2-F and -E (same architecture, same serve contract, drop-in). The first of our drafters to beat the reference incoai/GLM-5.3-Flash-DFlash2 outside the protocol's noise on identical hardware: 3.676 vs 3.632 mean acceptance at K=7, 3.697 vs 3.667 single-stream (K=4: 3.088 vs 3.136, still 1.5% short). See below.

Architecture (unchanged from -E / -F)

Training

• Warm-started from dflash2F; 70,000 steps over 736,675 self-generated samples (the 449,600 of -F + 291,020 new completions of never-before-generated prompts: real user chat, tool calling, hard math, code and STEM), 0.76 epoch, lr 1e-4 → 1e-5 (cosine), gamma-6.5 tail weighting, CE + top-20 KL, 8× NVIDIA B300, 27.4 h at 1.41 s/step.
• Prompts sampled from public instruction sets — the -F set (ultrachat_200k · MIT, OpenR1-Math-220k · Apache-2.0, OpenMathReasoning · CC-BY-4.0, OpenScienceReasoning-2 · CC-BY-4.0, OpenCodeReasoning / OpenCodeInstruct · CC-BY-4.0, evol-codealpaca-v1 · Apache-2.0) plus, new for -G, lmsys-chat-1m and WildChat-1M (real user prompts, English, moderation-filtered), NuminaMath-1.5, OpenThoughts3 (math / code / science prompts), xlam-function-calling-60k (tool-calling prompts) and Nemotron-Post-Training-v2 STEM prompts; every completion was regenerated by the target model itself (thinking ON, T=1.0 / top_p 0.95, ≤ 4,096 tokens) — no third-party model outputs, only prompts.
• The reference drafter incoai/GLM-5.3-Flash-DFlash2 was never a training input; it appears below only as a measured comparison.
• PROVENANCE.txt is the verbatim training record. Recipe, code, patches and every raw measurement: canada-quant/vllm-glm53-flash-sm121/drafter (eval kit, every raw measurement, cards).

Holdout evaluation — same hardware, same day

500 never-trained-on prompts, thinking ON, T=1.0 / top_p 0.95, max_tokens 1024, greedy drafts, TP=4 (c16; the c1 row is 100 prompts at concurrency 1). All rows below were measured on the same 8× B300 with the same vLLM build and the same target backend (FLASHINFER_MLA_SPARSE) (-G on 2026-09-25; -F, -E and incoai on 2026-09-23) — acceptance length shifts by a few hundredths between GPU generations (incoai reads 3.602–3.615 on H200 vs 3.632 on B300), so only same-hardware rows are compared.

Per-position acceptance (K=7, c16): 0.753 · 0.562 · 0.422 · 0.324 · 0.253 · 0.201 · 0.161 (incoai: 0.75 · 0.55 · 0.41 · 0.32 · 0.25 · 0.20 · 0.16; -F: 0.750 · 0.555 · 0.415 · 0.316 · 0.244 · 0.193 · 0.153).

Deep traces (same protocol with max_tokens 4096, K=7, c16): this drafter 3.478, incoai 3.473 — parity. Both drafters lose ~0.16–0.20 acceptance when the thinking trace runs to 4K tokens, and the 1K-token lead above does not carry that deep; quote both numbers. (Per-position @4K: 0.729 · 0.526 · 0.388 · 0.292 · 0.225 · 0.177 · 0.141 vs incoai 0.728 · 0.523 · 0.384 · 0.291 · 0.226 · 0.179 · 0.144.)

Honest read: +0.044 over the reference at K=7 and +0.030 single-stream, both outside the protocol's ±0.015 run-to-run noise — every draft position is at or above the reference. K=4 is still 1.5% short (−0.048): the reference keeps a small edge on the first two positions when only four are drafted. Against -F it is +0.050 at K=7 from 291K fresh, more diverse self-generated samples (real chat and tool-calling prompts the earlier sets never had). Fully ours (Apache-2.0, no NC/ND terms), trained on data we control, reproducible end-to-end.

DGX Spark (SM121): -E reproduced its H200 acceptance on 2× Spark within 0.3% (3.5788 c16 / 3.5970 c1 vs 3.568 / 3.585). -G is a drop-in for the same launcher and the Spark drafter of record since 2026-09-25: the H2H champion leg (2026-09-27; 702/702 requests, coherence-clean; tg32 d0 31.35±3.52 / c1@65k 15.34 quiescent — receipt: H2H_FINAL_VERDICT_2026-09-27.md, spark-cluster PR #27) and the live production serve both ran G (sha256-verified on both hosts). A same-rig Spark acceptance A/B vs the reference remains unmeasured (the acceptance table above is B300) — re-measure with the eval kit in the repo before quoting a Spark acceptance number.

Serving

Pairs with canada-quant/GLM-5.3-Flash-W4A16-MTP (the quant it was trained against) or BF16 GLM-5.3-Flash. vLLM speculative config:

{"method": "dflash", "model": "/models/GLM-5.3-Flash-DFlash2-G", "num_speculative_tokens": 7}

x86 (H100 / H200 / B300), upstream vLLM nightly ≥ 2026-09-08 (+ the two GLM-5.3-Flash DFlash2 bind-mount patches in the repo above):

vllm serve /models/glm53-flash-w4a16-mtp --served-model-name glm53 --tensor-parallel-size 4 --enable-expert-parallel \
  --block-size 64 --no-enable-prefix-caching \
  --speculative-config '{"method":"dflash","model":"/models/GLM-5.3-Flash-DFlash2-G","num_speculative_tokens":7}'

2× DGX Spark (SM121): prebuilt image + one-command launcher in canada-quant/vllm-glm53-flash-sm121 — the drafter is bind-mounted at runtime (DRAFTER_HOST_PATH=/models/GLM-5.3-Flash-DFlash2-G), no rebuild. launch_dflash2_tp2.sh in this repo is the same TP=2 launcher with this drafter as its default.

Hard constraints (all measured, not stylistic):

• num_speculative_tokens: 7 (= block_size − 1) is the trained and measured value. k=5 also boots on the SM121 image (2026-09-26, not benchmarked); an earlier version of this card said other counts boot-wedge the stack, which was wrong.
• mask_embedding.pt must sit next to the weights. Verify the boot log carries Loaded DFlash mask embedding for mask_token_id 154856 from mask_embedding.pt — absence means the mask was silently ignored; do not serve.
• The 9-tap config requires the serving stack to honor dflash_config.target_layer_ids of length 9 (upstream vLLM DFlash2 does — vllm-project/vllm#52816).
• Full-attention drafter layers: the target stack must accept FullAttentionSpec drafter KV in the GLM-5 KV fast path (the SM121 image above has it; upstream nightly needs the kv_cache_utils patch from the repo).

Memory cost — and when to use the built-in MTP head instead (added 2026-10-04)

This drafter's 8 layers use full attention, so each keeps a KV cache for the whole context; the incoai reference drafter keeps a 2,048-token sliding window. KV pools measured by the authors on 2× DGX Spark (fp8 KV):

That is about 3× less context per GiB of KV. A 9 GiB pin holds ≈450K tokens with -E / -F / -G — less than one 1M request — so 1M context on 2× Spark is validated only with the incoai drafter; with our drafters the validated contexts are 262K (8 GiB) and 800K (16 GiB).

• Use this drafter for short-context English and code, where its acceptance is highest.
• Use GLM-5.3-Flash's built-in BF16 MTP head (shipped in the W4A16 checkpoint; {"method":"mtp","num_speculative_tokens":3}) for long context, non-English text, or KV headroom.
• Measured by the authors (W4A16 + -G, 2× Spark, our SM121 image, single stream, llama-benchy pp2048/tg128): 31.4 / 30.7 / 15.3 / 10.0 tok/s at context depth 0 / 4K / 65K / 100K (2026-09-28; the 100K cell from 2026-09-26).
• Community-reported (NVIDIA developer forum, 2026-10-04; not measured by us): on eugr's b12x vLLM build with the GPU clock capped at 1700 MHz, the W4A16 weights with the MTP head held 31.6 / 29.8 / 25.4 / 31.0 tok/s at the same depths; with -G usable context fell from ≈930K to ≈330K tokens and decode was 20–30% slower on non-English text (faster on English code).
• Our own same-pair comparison on 2× Spark (drafter vs MTP, plus eugr's b12x build) has been running since 2026-10-04 on a bilingual generation workload; results will be published in canada-quant/vllm-glm53-flash-sm121.

Provenance

• Files: model.safetensors (sha256 0d2a3eebfc532adfe9d092f0b972780592be8b587cf0fcc871d6c936c0b353f6), config.json (699bdf29…c51778, identical to -E / -F), mask_embedding.pt (291c7243…ec82ec), PROVENANCE.txt.
• Every number on this card comes from sha256-pinned raw eval JSONs banked in the repo (angelspec-glm53/runs/all/dflash2G-*.json incl. dflash2G-k7-mt4096.json, incoai-k7-mt4096-b300.json, dflash2F-*.json, dflash2E-k7-b300.json, incoai-k7-b300.json, incoai-k4-b300.json, incoai-k7-c1-b300.json).

References

• DFlash2 method: z-lab/dflash · arXiv 2602.06036 · inco.ai blog
• Upstream vLLM DFlash2 integration: vllm-project/vllm#52816
• Trainer: Tencent/AngelSpec (Apache-2.0)
• Target model: zai-org/GLM-5.3-Flash · our quant: canada-quant/GLM-5.3-Flash-W4A16-MTP
• Reference drafter (comparison only): incoai/GLM-5.3-Flash-DFlash2 (CC-BY-NC-ND-4.0)