HybridDiffusion-9B

Model overview

HybridDiffusion-9B is an 8.95B-parameter text-generation checkpoint produced by applying HybridDiffusion's autoregressive-to-diffusion conversion recipe to Qwen3.5-9B. A single set of weights supports AR-Trust verified decoding, Diffusion-Trust parallel denoising, and ordinary causal generation through the HybridDiffusion runtime.

The native mask ID is 248077. It belongs to the padded model vocabulary and is intentionally not exposed as a tokenizer token.

Quick Start Guide

HybridDiffusion inference uses the SGLang and FlashInfer implementations bundled with the HybridDiffusion repository. The checkpoint uses a standard Hugging Face layout and is downloaded automatically by the serving runtime.

1. Install the runtime

git clone https://github.com/yuchen-zhu-zyc/HybridDiffusion.git
cd HybridDiffusion/eval

HYBRID_DIFFUSION_CACHE_ROOT=/persistent/hybrid-diffusion-cache \
  bash scripts/setup_eval_env.sh

The one-shot installer creates the evaluation environment and places reusable Hugging Face, FlashInfer, Triton, and extension caches below HYBRID_DIFFUSION_CACHE_ROOT.

2. Start a server

Start one decoding mode. Each command below serves the same checkpoint and uses port 30000 unless PORT is overridden.

# AR-Trust: noisy-stream drafting with clean-stream verification.
scripts/serve.sh self-spec yuchen-zhu-zyc/HybridDiffusion-9B

# Diffusion-Trust: parallel block denoising.
scripts/serve.sh diffusion yuchen-zhu-zyc/HybridDiffusion-9B

# Causal reference: ordinary left-to-right decoding.
scripts/serve.sh causal yuchen-zhu-zyc/HybridDiffusion-9B

To run more than one server, assign a different PORT and GPU to each process, for example PORT=30001 CUDA_VISIBLE_DEVICES=1 ....

3. Query the OpenAI-compatible API

curl http://localhost:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "yuchen-zhu-zyc/HybridDiffusion-9B",
    "messages": [{"role": "user", "content": "Solve: 17 × 23."}],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 20,
    "max_tokens": 512,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

The installed environment also includes the OpenAI Python client:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="yuchen-zhu-zyc/HybridDiffusion-9B",
    messages=[{"role": "user", "content": "Solve: 17 × 23."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=512,
    extra_body={
        "top_k": 20,
        "chat_template_kwargs": {"enable_thinking": False},
    },
)
print(response.choices[0].message.content)

Decoding modes

AR-Trust uses speculation horizon N=4 and the Exact-Truncated verification policy. Diffusion-Trust uses runtime block_size: 3, which implements logical block size B=4 under HybridDiffusion's shifted decoding rule. See the evaluation guide for the full configuration and sampling interface.

The API example controls request-level sampling. The released AR-Trust proposal and verification law uses temperature 1.0, top-k 50, and top-p 0.95; Diffusion-Trust uses threshold 0.95, temperature 1.0, top-k 20, and top-p 0.95 in its algorithm configuration.

Benchmark results

The table reproduces Table 1 of the HybridDiffusion paper. HybridDiffusion's AR-Trust and Diffusion-Trust columns use the same checkpoint. Bold marks the best diffusion LLM result among the public comparison models in each row; * denotes a result identified in the paper as potentially under-reported. — means the source did not report that result.

Mercury-2 is commercial and Qwen3.5-9B is the autoregressive source checkpoint. Neither column is included when selecting the bolded open-source diffusion-LLM result.

Reproduce evaluation

With a server running on port 30000, launch the complete evaluation suite from the repository's eval directory:

PORTS=30000 \
TASKS="gsm8k math500 ifeval humaneval mbpp arc_c gpqa aime2024 aime2025 mmlu mmlu_pro lcb" \
  scripts/evaluate.sh

The evaluator writes generations, scores, executed commands, and a run manifest below HYBRID_DIFFUSION_CACHE_ROOT. See the evaluation README for task-specific requirements and full-evaluation settings.

Citation

@article{zhu2026flare,
  title   = {FLARE: Diffusion for Hybrid Language Model},
  author  = {Yuchen Zhu and Jing Shi and Chongjian Ge and Hao Tan and
             Yiran Xu and Wanrong Zhu and Jason Kuen and Koustava Goswami and
             Rajiv Jain and Yongxin Chen and Molei Tao and Jiuxiang Gu},
  journal = {arXiv preprint arXiv:2606.01774},
  year    = {2026}
}

Acknowledgements

HybridDiffusion-9B is derived from Qwen3.5-9B. The serving stack builds on SGLang and FlashInfer.