Qwen3.8-27B-DSpark
This repository contains a DSpark draft model for Qwen/Qwen3.8-27B-FP8. This version is optimized for medium-effort reasoning and agentic workloads. It is not a standalone language model: it runs alongside the target model in a speculative-decoding server, proposes 7 tokens per round, and leaves final verification to the target.
Training at a glance
Training broadly follows DeepSeek's DSpark recipe and used approximately 1.04B output tokens and 2.627B anchor exposures, broken down by reasoning effort:
| Reasoning effort | # output tokens | # anchor exposures |
|---|---|---|
off | ~110.5M | ~299.0M |
low | ~124.8M | ~323.4M |
medium | ~560.5M | ~1.391B |
xhigh | ~244.2M | ~614.1M |
| Total | ~1.040B | ~2.627B |
At this data scale, across these 12 workloads, this model's request-macro / proposal-weighted EAL exceeds RadixArk DSpark v2 by 0.676 / 0.609 at medium and 0.073 / 0.044 at xhigh.
Model details
- Training target: Qwen/Qwen3.8-27B-FP8
- Draft parameters: ~1.86B
- Draft weight dtype: BF16
- Training/verification block width: 8 tokens (7 draft proposals)
- Hidden size: 5,120
- Transformer: 5 sliding-window (width 2048) attention layers
- Attention: GQA with 32 query heads and 8 key/value heads
- Target feature layers (zero-based): 5, 19, 33, 47, 61
- Markov head:
VanillaMarkov, rank 256 - Maximum position embeddings: 262,144
Acceptance length
Each cell reports request-macro / proposal-weighted EAL; higher is better. For τ³, both the agent and FP8 user simulator use the effort stated by the table.
Medium effort
| Category | Workload | Prompts | RadixArk DSpark v2 | This model |
|---|---|---|---|---|
| Chat | MT-Bench | 80 | 3.6115 / 2.9688 | 4.2590 / 3.4679 |
| Chat | Arena-Hard-v2 | 750 | 3.4796 / 3.2005 | 4.0044 / 3.6402 |
| Chat (ZH) | AlignBench | 683 | 3.1208 / 2.7952 | 3.4843 / 3.0427 |
| Math | GSM8K | 1,319 | 5.1381 / 5.0211 | 6.2282 / 6.0783 |
| Math | MATH-500 | 500 | 5.0465 / 4.5891 | 5.6797 / 5.0241 |
| Reasoning | GPQA-Diamond | 198 | 4.2926 / 4.1066 | 5.0217 / 4.7636 |
| Code | HumanEval | 164 | 5.1694 / 5.0687 | 6.2000 / 6.1004 |
| Code | MBPP | 257 | 4.2923 / 4.3148 | 5.2898 / 5.2560 |
| Code (ZH) | NaturalCodeBench-ZH | 140 | 4.8401 / 4.7367 | 5.5242 / 5.4396 |
| Agent | AutomationBench Operations/Public | 100 | 3.8366 / 3.3787 | 4.4280 / 3.7993 |
| Agent | BFCL v4 multi-turn | 200 | 4.5050 / 4.0989 | 4.9922 / 4.6943 |
| Agent | τ³ telecom/base | 114 | 3.6718 / 3.3192 | 4.0050 / 3.5968 |
| 12-workload equal mean | 4,505 | 4.2503 / 3.9665 | 4.9264 / 4.5753 |
Xhigh effort
| Category | Workload | Prompts | RadixArk DSpark v2 | This model |
|---|---|---|---|---|
| Chat | MT-Bench | 80 | 3.7028 / 3.1899 | 3.8575 / 3.2986 |
| Chat | Arena-Hard-v2 | 750 | 3.3082 / 3.1261 | 3.3142 / 3.1317 |
| Chat (ZH) | AlignBench | 683 | 3.2091 / 2.9249 | 3.1502 / 2.8814 |
| Math | GSM8K | 1,319 | 5.4651 / 4.6406 | 5.2977 / 4.5819 |
| Math | MATH-500 | 500 | 5.0944 / 4.3434 | 5.0485 / 4.2586 |
| Reasoning | GPQA-Diamond | 198 | 3.6749 / 3.5643 | 3.6917 / 3.6054 |
| Code | HumanEval | 164 | 4.1540 / 3.8983 | 4.1382 / 3.8639 |
| Code | MBPP | 257 | 4.5106 / 4.0360 | 4.5590 / 4.0351 |
| Code (ZH) | NaturalCodeBench-ZH | 140 | 3.4532 / 3.3326 | 3.4290 / 3.3337 |
| Agent | AutomationBench Operations/Public | 100 | 3.9347 / 3.6934 | 4.4301 / 3.9239 |
| Agent | BFCL v4 multi-turn | 200 | 4.0618 / 3.6487 | 4.3000 / 3.8319 |
| Agent | τ³ telecom/base | 114 | 3.7253 / 3.2859 | 3.9483 / 3.4718 |
| 12-workload equal mean | 4,505 | 4.0245 / 3.6403 | 4.0970 / 3.6848 |
Evaluation protocol
- Drafts: the released
Qwen3.8-27B-DSparkweights andRadixArk/Qwen3.8-27B-DSpark@b9a5dbd - Sampling: thinking enabled, temperature 1.0, top-p 0.95, top-k 20
- Recorded draft seed bases: this model 42, RadixArk 980406.
Serving with SGLang
Use an SGLang build containing Qwen3.8 DSpark support:
pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"
python -m sglang.launch_server \
--model-path Qwen/Qwen3.8-27B-FP8 \
--speculative-algorithm DSPARK \
--speculative-draft-model-path yukimakoto/Qwen3.8-27B-FP8-DSpark \
--speculative-dspark-block-size 7