Qwen3.5-2B DFlash Draft Model

A DFlash speculative-decoding draft model for Qwen3.5-2B .

What is DFlash?

DFlash is a lightweight speculative-decoding draft method. Instead of decoding one token at a time, the draft model proposes a whole block of candidate tokens in a single forward pass, conditioned on hidden states captured from several intermediate layers of the target model. The target model then verifies the block in parallel and accepts the longest correct prefix. This lets the system emit multiple tokens per target forward pass while keeping the output identical to greedy target decoding, which speeds up inference when decoding is memory-bandwidth bound.

Model

Files

• model.safetensors — draft model weights
• config.json — draft model config

Benchmark

Eval config

• Backend: sglang
• Decoding: greedy, temperature = 0, max_tokens = 2048
• Metric: accept length (AL) = Σ completion_tokens / Σ verify_ct (tokens emitted per target verify pass)

Per dataset

Overall

DFlash 是什么?

DFlash 是一种轻量级的投机解码(speculative decoding)草稿方法。草稿模型不再逐 token 解码,而是借助从目标模型多个中间层捕获的隐状态,在一次前向中一次性提出一整块(block)候选 token;随后目标模型并行校验该 block,并接受最长的正确前缀。因此在保持与目标模型贪心解码完全一致输出的前提下,每次目标前向可产出多个 token ,进而在访存瓶颈时提高推理速度。

模型信息

文件说明

• model.safetensors — 草稿模型权重
• config.json — 草稿模型配置

评测结果

评测配置

• 后端: sglang
• 解码: greedy, 温度 = 0, max_tokens = 2048
• 指标: 接受长度 (AL) = Σ completion_tokens / Σ verify_ct(每次目标校验前向产出的 token 数)

分数据集

总览