TEV Draft Model for Qwen3-Coder-30B-A3B-Instruct
This repository contains the ExitTrain-trained DFlash draft model for Qwen/Qwen3-Coder-30B-A3B-Instruct, released with TEV: Exit-Guided Speculative Decoding.
GitHub | DDTree + TEV guide | Paper
This is a draft model, not a standalone language model. It must be paired with the matching Qwen3-Coder target model in a speculative decoding system.
Model details
| Field | Value |
|---|---|
| Target model | Qwen/Qwen3-Coder-30B-A3B-Instruct |
| Target architecture | Mixture of Experts |
| Draft architecture | DFlashDraftModel |
| Training method | ExitTrain |
| Tree builder and verifier | DDTree + TEV |
| Draft block size | 16 |
| Target layers | 48 |
| Target feature layers | [1, 12, 23, 34, 45] |
| Precision | BF16 |
ExitTrain trains the drafter on its realized finite-budget draft trees. Target probabilities are evaluated under the actual tree-parent contexts, and target-derived exit feedback is used to improve where the bounded tree allocates its nodes.
Quick start with DDTree + TEV
git clone https://github.com/hsj576/TEV.git
cd TEV/ddtree
pip install -r requirements.txt
export TARGET_MODEL=Qwen/Qwen3-Coder-30B-A3B-Instruct
export DRAFT_MODEL=husj576/TEV-qwen3-coder-30B-A3B
bash scripts/run_ddtree_tev_smoke.sh
The target, draft model, draft tree, and KV cache must fit on the same GPU in the current reference implementation.
Direct HumanEval benchmark
torchrun --nproc_per_node=1 --master_port=29700 benchmark.py \
--model-name-or-path Qwen/Qwen3-Coder-30B-A3B-Instruct \
--draft-name-or-path husj576/TEV-qwen3-coder-30B-A3B \
--dataset humaneval \
--max-samples 164 \
--max-new-tokens 2048 \
--temperature 1.0 \
--tree-budget 64 \
--save-path runs/qwen3_coder_30b/humaneval__t1_0__tb64.pt
Without --skip-baseline-dflash, the benchmark also runs autoregressive decoding and block-parallel DFlash as reference paths.
Usage notes
- Use this checkpoint only with
Qwen/Qwen3-Coder-30B-A3B-Instruct. - The released evaluation path uses the Qwen chat template with thinking disabled.
- At positive temperature, the DDTree branch uses TEV sampling. At temperature
0, it uses the equivalent greedy tree-follow path. - The current reference implementation targets batch-one decoding.
flash-attnis required by the draft model. The target uses PyTorch SDPA when DDTree's custom tree mask is enabled.- The Qwen3-Coder target requires substantially more GPU memory than Qwen3-4B/8B. Reduce tree budget and maximum generation length when memory is limited.