Qwen3-Coder-30B-A3B-DFlash

Paper | GitHub | Blog

DFlash is a novel speculative decoding method that utilizes a lightweight block diffusion model for drafting. It enables efficient, high-quality parallel drafting that pushes the limits of inference speed.

This model is the drafter component. It must be used in conjunction with the target model Qwen/Qwen3-Coder-30B-A3B-Instruct.

📊 Training Data & Efficiency

Qwen3-Coder-30B-A3B-DFlash is trained on 289K samples, composed of:

• Code split from nvidia/Nemotron-Post-Training-Dataset-v2
• theblackcat102/evol-codealpaca-v1
• Approximately 2.8K Cline execution traces collected by ourselves

Despite being trained on significantly less data, DFlash already outperforms EAGLE-3 in inference acceleration. In comparison, lmsys/SGLang-EAGLE3-Qwen3-Coder-30B-A3B-Instruct-SpecForge is trained on the open-perfect-blend dataset with 1.4M samples, nearly 5× more data than DFlash.

This result highlights the training efficiency and scalability of DFlash, and suggests that further scaling the training data can unlock even greater acceleration gains.

🚀 Quick Start

SGLang

Installation

uv pip install "git+https://github.com/sgl-project/sglang.git@refs/pull/20547/head#subdirectory=python"

Launch Server

# Optional: enable schedule overlapping (experimental, may not be stable)
# export SGLANG_ENABLE_SPEC_V2=1
# export SGLANG_ENABLE_DFLASH_SPEC_V2=1
# export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1

python -m sglang.launch_server \
    --model-path Qwen/Qwen3-Coder-30B-A3B \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path z-lab/Qwen3-Coder-30B-A3B-DFlash \
    --tp-size 1 \
    --dtype bfloat16 \
    --attention-backend fa3 \
    --mem-fraction-static 0.75 \
    --trust-remote-code

Usage

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="Qwen/Qwen3-Coder-30B-A3B",
    messages=[{"role": "user", "content": "Write a quicksort in Python."}],
    max_tokens=2048,
    temperature=0.0,
)
print(response.choices[0].message.content)

vLLM

Installation

uv pip install vllm
uv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly

Launch Server

vllm serve Qwen/Qwen3-Coder-30B-A3B \
  --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3-Coder-30B-A3B-DFlash", "num_speculative_tokens": 15}' \
  --attention-backend flash_attn \
  --max-num-batched-tokens 32768

Usage

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="Qwen/Qwen3-Coder-30B-A3B",
    messages=[{"role": "user", "content": "Write a quicksort in Python."}],
    max_tokens=2048,
    temperature=0.0,
)
print(response.choices[0].message.content)

Transformers

Installation

pip install transformers==4.57.3 torch==2.9.0 accelerate

Inference

from transformers import AutoModel, AutoModelForCausalLM, AutoTokenizer

model = AutoModel.from_pretrained(
    "z-lab/Qwen3-Coder-30B-A3B-DFlash", 
    trust_remote_code=True, 
    dtype="auto", 
    device_map="cuda:0"
).eval()

target = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-Coder-30B-A3B-Instruct", 
    dtype="auto", 
    device_map="cuda:0"
).eval()

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-Coder-30B-A3B-Instruct")
prompt = "Please provide a Python implementation of the Bubble Sort algorithm."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generate_ids = model.spec_generate(
    input_ids=model_inputs["input_ids"], 
    max_new_tokens=2048, 
    temperature=0.0, 
    target=target, 
    stop_token_ids=[tokenizer.eos_token_id]
)

print(tokenizer.decode(generate_ids[0], skip_special_tokens=True))

Evaluation

DFlash consistently achieves high acceptance lengths and speedups across different concurrency levels. DFlash achieves similar acceptance length for both the bfloat16 target model and its FP8 variant Qwen3-Coder-30B-A3B-Instruct-FP8. All experiments are conducted using SGLang on a single B200 GPU.

We use a block size of 16 during speculation. You can specify different block size during inference by passing --speculative-num-draft-tokens arguments when launch the server.

HumanEval

MBPP

LiveCodeBench

Acknowledgement

We are grateful to Yotta Labs for their compute support in training this draft model.

Citation

If you find DFlash useful for your research or applications, please cite our project.

@misc{chen2026dflash,
  title         = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author        = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  year          = {2026},
  eprint        = {2602.06036},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2602.06036}
}