Qwen3-4B DSpark Speculator (Block Size 16)
This repository contains a DSpark speculative decoding draft model trained for
Qwen/Qwen3-4B. It proposes up to 16
tokens per decoding iteration and is intended for use with the
vllm-project/speculators
implementation and vLLM.
This is an experimental baseline checkpoint. It was trained for one epoch with the standard cross-entropy and total-variation objective; it was not trained with D-PARD.
Model details
- Architecture: DSpark draft model
- Verifier:
Qwen/Qwen3-4B - Speculative block size: 16
- Draft transformer layers: 5
- Verifier hidden-state layers: 1, 9, 17, 25, 33
- Markov head: vanilla, rank 256
- Confidence head: enabled with Markov features
- Default proposal method: greedy
- Precision: bfloat16
- Draft vocabulary size: 151,936
- Parameters file size: 1.23 GB
Training
The model was trained from the speculators main branch at commit
ae96c74c0771ca4fb920df1279f41f60e54db2cc.
- Epochs: 1
- Global steps: 244,909
- World size: 4
- Sequence length: 8,192
- Optimizer: AdamW
- Learning rate: 6e-4
- Weight decay: 0
- Scheduler: cosine with 4% warmup
- Seed: 42
- Loss:
0.1 * CE + 0.9 * TV - Position weighting: fixed exponential decay, gamma 4.0
speculators: 0.7.0.dev150- Training vLLM: 0.26.0
- Transformers: 5.12.0
- PyTorch: 2.11.0
The complete invocation is included in train_command.txt.
Validation results
| Metric | Value |
|---|---|
| Loss | 0.2620 |
| Cross-entropy loss | 0.4829 |
| Total-variation loss | 0.1064 |
| Acceptance rate | 0.4361 |
| Acceptance length | 4.5802 |
| Full accuracy | 0.4958 |
Speculator benchmark results
The checkpoint was evaluated with the D-PARD evaluation implementation on
RedHatAI/speculator_benchmarks, with temperature 0, maximum generation length
128, maximum concurrency 128, and up to 200 requests per category.
| Category | Acceptance length |
|---|---|
| HumanEval | 4.9515 |
| Math reasoning | 6.2743 |
| QA | 4.0288 |
| Question | 4.0356 |
| RAG | 3.9408 |
| Summarization | 2.8349 |
| Tool call | 3.8006 |
| Translation | 4.0431 |
| Writing | 4.0295 |
| Weighted overall | 4.4054 |
The weighted overall result is computed as
1 + total accepted speculative tokens / total draft iterations.
For context, under the same dataset and sampling settings,
RedHatAI/Qwen3-4B-speculator.dspark using 8 speculative tokens achieved a
weighted acceptance length of 4.7543. These numbers are not a strict
architecture-controlled comparison because the models use different block
sizes (16 versus 8).
Usage
Use this repository as the speculative model for Qwen/Qwen3-4B through a
version of vLLM and speculators that supports DSpark. The verifier model and
tokenizer must remain Qwen/Qwen3-4B, and the configured speculative-token
count is 16.
Limitations
- This is a research checkpoint for speculative decoding, not a standalone language model.
- It inherits the behavior, limitations, and license considerations of the verifier model.
- Acceptance and speedup depend on prompts, hardware, batch size, vLLM version, and serving configuration.
- A larger speculative block does not guarantee higher average acceptance or end-to-end throughput.