GLM-5.3-Flash-DFlash2-GGUF
Community GGUF of Inco AI's DFlash 2 draft model for GLM-5.3-Flash.
This is not a standalone language model. It only drafts tokens for a GLM-5.3-Flash target under speculative decoding.
Spark serve recipe: vcruz305/GLM-5.3-Flash-DFlash2-DGX-Spark-recipe
Source and attribution
- Original weights: incoai/GLM-5.3-Flash-DFlash2
- Target model: zai-org/GLM-5.3-Flash
- Method: DFlash 2: Keep Drafting Parallel
- Code: z-lab/dflash
License follows the original: CC BY-NC-ND 4.0. For commercial use, contact contact@inco.ai.
If you use this GGUF, please cite Inco AI's DFlash 2 writeup and the DFlash paper:
@misc{inco2026dflash2,
title = {{DFlash 2: Keep Drafting Parallel}},
author = {{Inco AI}},
year = {2026},
month = {August},
url = {https://inco.ai/blog/dflash2/}
}
@inproceedings{chen2026dflash,
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
booktitle = {International Conference on Machine Learning (ICML)},
year = {2026}
}
Files
| File | Type | Size | Notes |
|---|---|---|---|
GLM-5.3-Flash-DFlash2-BF16.gguf | BF16 GGUF | 2.191 GiB (2,352,022,432 B) | 81 tensors, arch dflash |
GGUF metadata from this convert: dflash.block_size=8, conv_kernel_size=2, conv_group_size=16, selector_rank=256, selector_top_k=16, target_layers=[6,15,25,34,43]. Vocab is the GLM-5.3 tokenizer (154880).
Q8_0 / Q4_K_M drafts were measured on a Spark and are not in this repo.
Run
Needs llama.cpp with DFlash 2 (grouped dynamic conv + candidate selector). That is ggml-org/llama.cpp#27342, also on vcruz305/llama.cpp main / glm5next-mtp at 6f5ac9a (+ aarch64 cmath 4a06ec6). Pair with a GLM-5.3-Flash target GGUF such as vcruz305/GLM-5.3-Flash-GGUF.
--spec-draft-n-max clamps to 7 (block_size - 1). Do not combine with --spec-type draft-mtp on the same server.
hf download vcruz305/GLM-5.3-Flash-DFlash2-GGUF GLM-5.3-Flash-DFlash2-BF16.gguf --local-dir GLM-5.3-Flash-DFlash2-GGUF
llama-server \
-m GLM-5.3-Flash-Q2_K.gguf \
-md GLM-5.3-Flash-DFlash2-GGUF/GLM-5.3-Flash-DFlash2-BF16.gguf \
--spec-type draft-dflash --spec-draft-n-max 7 --spec-draft-p-min 0.30 \
-fa on -ctk q8_0 -ctv q8_0 --jinja \
-c 98304 -np 1 --no-kv-unified --fit off
Measured (one DGX Spark GB10, 2026-08-27)
Target: GLM-5.3-Flash-Q2_K.gguf. Tool: llama-speculative-simple, greedy, seed 42.
Unique prompt "The capital of France is" (n=64, FA on, no q8 KV): 17.58 t/s, accept 31.41%.
Repetitive bench file, FA + q8 KV, -c 2048, n=128:
| draft | n_max | t/s | accept |
|---|---|---|---|
| MTP-3 control | 3 | 28.23 | 73.2% |
| DFlash2 BF16 | 7 | 41.40 | 94.4% |
| DFlash2 Q4_K_M | 7 | 43.43 | 94.4% |
Do not quote 94% as a model score — that file is a repeated sentence.
Ctx ladder (Q4_K_M, n_max=7, p_min=0.30, FA+q8, n=64):
| ctx | t/s | accept |
|---|---|---|
| 8,192 | 39.58 | 89.9% |
| 32,768 | 39.53 | 89.9% |
| 65,536 | 39.71 | 89.9% |
| 98,304 | 38.86 | 89.9% |
96k is the last measured OK (114,820 MiB of 124,610). 114k/128k not re-run for DFlash2.
Convert
python convert_hf_to_gguf.py incoai/GLM-5.3-Flash-DFlash2 \
--target-model-dir zai-org/GLM-5.3-Flash-BF16 \
--outtype bf16 \
--outfile GLM-5.3-Flash-DFlash2-BF16.gguf
--target-model-dir is tokenizer + config.json only. Converted with vcruz305/llama.cpp 6f5ac9a.