GLM-5.3-NVFP4-DFlash
This repository contains a DFlash draft model for modal-labs/GLM-5.3-NVFP4. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.
DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.
Quick Start
Serve with SGLang v0.5.19 or later:
python -m sglang.launch_server \
--model-path modal-labs/GLM-5.3-NVFP4 \
--quantization modelopt_fp4 \
--tp-size 4 \
--trust-remote-code \
--speculative-algorithm DFLASH \
--speculative-draft-model-path modal-labs/GLM-5.3-NVFP4-DFlash \
--speculative-dflash-block-size 8 \
--speculative-draft-model-quantization unquant \
--speculative-draft-attention-backend fa4 \
--host 0.0.0.0 \
--port 30000
Keep the draft model unquantized. Quantizing it lowers the accept length.
License
Distributed under Z.ai's GLM-5.3 License, inherited from the target model.
Citation
If you find DFlash useful, please cite the original paper:
@article{chen2026dflash,
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
journal = {arXiv preprint arXiv:2602.06036},
year = {2026}
}