GLM-5.2-FP8-DFlash

Paper | DFlash GitHub | SpecForge

DFlash block-diffusion speculative-decoding drafter for GLM-5.2-FP8 (743B MoE, 39B active). Standard DFlash method (no extensions), trained with SpecForge on the paper-specified data recipe: ~800K samples of Nemotron-Post-Training-v2 + CodeAlpaca (code / math / chat), all responses regenerated by GLM-5.2-FP8, 6 epochs — directly comparable to the DFlash paper and the z-lab drafter series.

Quick Start (SGLang)

python -m sglang.launch_server \
    --model-path zai-org/GLM-5.2-FP8 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path UCloud-org/GLM-5.2-FP8-DFlash \
    --speculative-num-draft-tokens 16 \
    --tp-size 8 \
    --trust-remote-code

vLLM v0.20.1+ has native DFlash support and reads this checkpoint directly, no conversion needed. Not yet runtime-verified on our infrastructure; the results below were produced via SGLang.

Evaluation

Mean accepted length & end-to-end speedup

Measured on live SGLang serving (concurrency 1, greedy decoding unless noted).

Built-in MTP baseline uses the official GLM-5.2 recipe (EAGLE, steps 5 / topk 1 / draft tokens 6). This drafter is code/math-optimized: it delivers 1.4-1.5x over the (already strong) built-in MTP on code and math workloads, while chat and Chinese-language workloads slightly favor built-in MTP (see Limitations).

ceval (Chinese) is the weakest domain — the training corpus is English-dominant (see Limitations).

Also mirrored on ModelScope: UCloud-AILab/GLM-5.2-FP8-DFlash.

Training Details

• Target model: GLM-5.2-FP8 (hidden 6144, 78 layers; drafter conditions on target layers [1, 20, 38, 56, 75])
• Drafter: 5-layer block-diffusion transformer, block_size 16, 3.7B total parameters (1.8B independently trained; embed/lm_head reused from target, frozen, not trained, included for standalone inference)
• Data: JessieWei/GLM-5.2-FP8-nemotron-codealpaca — Nemotron-Post-Training-v2 + CodeAlpaca, ~800K samples (paper-specified recipe), all responses regenerated by GLM-5.2-FP8 (non-thinking mode), max_length 3072
• Recipe: 6 epochs, AdamW with update clipping (StableAdamW-style), lr 6e-4 cosine (4% warmup), grad-clip 1.0, num_anchors 512, loss_decay_gamma 7, pure cross-entropy (standard DFlash loss)
• Framework: SpecForge (offline hidden-state pipeline), FSDP2

Limitations

• Trained on non-thinking-mode regenerated data; speedup under thinking-mode inference has not been evaluated yet.
• Trained with max_length 3072: accept length degrades on very long prompts. For long-context workloads, benchmark against the built-in MTP before choosing.
• Training corpus is English-dominant: acceptance length on Chinese-language workloads is lower (ceval AL 2.98 vs 4.4-8.0 on English benchmarks).

Acknowledgements

DFlash (z-lab), SpecForge / SGLang (sgl-project). GLM-5.2 by Zhipu AI.

Citation

If you use this model, please cite:

@misc{ucloud2026glm52dflash,
  title        = {GLM-5.2-FP8-DFlash: A DFlash Speculative-Decoding Drafter for GLM-5.2-FP8},
  author       = {Wei, Xiaojun and {UCloud AILab}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/UCloud-org/GLM-5.2-FP8-DFlash}}
}

This model is trained with the DFlash method — please also cite:

@misc{chen2026dflash,
  title         = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author        = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  year          = {2026}, eprint = {2602.06036}, archivePrefix = {arXiv},
  primaryClass  = {cs.CL}, url = {https://arxiv.org/abs/2602.06036}
}