Qwen3-32B-Thinking-speculator.eagle3
Model Overview
- Verifier: Qwen3-32B
- Speculative Decoding Algorithm: EAGLE-3
- Model Architecture: Eagle3Speculator
- Release Date: 3/12/2026
- Version: 1.0
- Model Developers: RedHat
This model is a speculator model designed for use with Qwen/Qwen3-32B, based on the EAGLE-3 speculative decoding algorithm. It was trained using the speculators library on a combination of the Magpie-Align/Magpie-Pro-300K-Filtered and the HuggingFaceH4/ultrachat_200k datasets.
This model is optimized for reasoning and chain-of-thought workloads and is meant to be used with thinking enabled.
It must be used with the Qwen3-32B chat template, specifically through the /chat/completions endpoint.
Use with vLLM
vllm serve Qwen/Qwen3-32B \
-tp 1 \
--speculative-config '{
"model": "RedHatAI/Qwen3-32B-Thinking-speculator.eagle3",
"num_speculative_tokens": 5,
"method": "eagle3"
}'
Evaluations
Model / run: Qwen3-32B-Thinking-speculator.eagle3
vLLM: 0.17.0rc1.dev122+g26bd43b52.d20260306.precompiled
Training data: Magpie + UltraChat; responses from the Qwen/Qwen3-32B model (with reasoning enabled).
Configuration
- Model: Qwen3-32B-Thinking-speculator.eagle3
- Data: Magpie+ ultrachat — responses from Qwen3-32B model (reasoning)
- temperature: 0.0
- vLLM version: 0.17.0rc1.dev122+g26bd43b52.d20260306.precompiled
- backend: vLLM chat_completions
- rate-type: throughput
- max-seconds per run: 300
- hardware: 8× GPU (tensor parallel 8)
- Benchmark data: RedHatAI/speculator_benchmarks
- vLLM serve: --no-enable-prefix-caching, --max-num-seqs 64, --enforce-eager
Command
GUIDELLM__PREFERRED_ROUTE="chat_completions" \
GUIDELLM__MAX_CONCURRENCY=128 \
guidellm benchmark \
--target "http://localhost:8000/v1" \
--data "RedHatAI/speculator_benchmarks" \
--data-args '{"data_files": "HumanEval.jsonl"}' \
--rate-type throughput \
--max-seconds 300 \
--backend-args '{"extra_body": {"chat_completions": {"temperature": 0.0}}}'
GuideLLM interface changed, so for compatibility with the latest version (v0.6.0), please use the following command:
GUIDELLM__PREFERRED_ROUTE="chat_completions" \
guidellm benchmark \
--target "http://localhost:8000/v1" \
--data "RedHatAI/speculator_benchmarks" \
--data-args '{"data_files": "HumanEval.jsonl"}' \
--profile sweep \
--max-seconds 1800 \
--output-path "my_output.json" \
--backend-args '{"extras": {"body": {"temperature":0.6, "top_p":0.95, "top_k":20}}}'