Qwen3.8-27B-MTP-MLX-bf16
This is not a standalone model. It holds only the Multi-Token Prediction (MTP) drafter head, so it carries no token embeddings and no
lm_headand it cannot generate text alone. Load it as the draft model beside a matching target, which supplies both.
Qwen ships this head inside
Qwen/Qwen3.8-27B as 15 tensors under
the mtp. prefix, but the MLX converter deletes them when it builds the main
model, so the head is published separately and the runtime loads it as its
own model with model_type: qwen3_5_mtp.
Use with mlx-vlm
mlx_vlm generate \
--model vvsotnikov/Qwen3.8-27B-MLX-8bit \
--draft-model vvsotnikov/Qwen3.8-27B-MTP-MLX-bf16 \
--prompt "Write a quicksort in Python." \
--max-tokens 256 --temperature 0.6 --enable-thinking
For local weights:
mlx_vlm generate \
--model /path/to/target-model \
--draft-model /path/to/Qwen3.8-27B-MTP-MLX-bf16 \
--prompt "Write a quicksort in Python." \
--max-tokens 256 --temperature 0.6 --enable-thinking
--draft-kind mtp is detected from model_type, so you do not need to pass
it. The drafter proposes tokens each step while the target verifies them, and
only accepted tokens reach the output, so quality follows the target rather
than the drafter.
Model Details
- Model type:
qwen3_5_mtp - MTP block size:
3 - Target architecture: Qwen3.8-27B
- Precision: bfloat16, unquantized
- Runtime: MLX /
mlx-vlm0.6.8 - Format: Safetensors with MLX-compatible config and tokenizer files
How this was produced
python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
--model Qwen/Qwen3.8-27B --output Qwen3.8-27B-MTP-MLX-bf16
The tool reads the 15 mtp. tensors out of the base checkpoint, strips that
prefix, and adds 1.0 to every RMSNorm weight, because Qwen stores the norm
scale minus one.
Verification
| Check | Result |
|---|---|
model_type | qwen3_5_mtp |
block_size | 3 |
| Tensors | 15, all bfloat16 |
| RMSNorm shift | applied exactly once, checked against the source |
| Acceptance | see the quantized drafters; bf16 drafts at least as well |
Pairing rules
Use a drafter and a target that come from ONE checkpoint, because the drafter binds to the target's embeddings at runtime, so a mismatched pair either fails a hidden-size check or drafts badly. This release publishes these drafters, so you can trade drafter size against acceptance:
License and attribution
The weights derive from Qwen/Qwen3.8-27B
under Apache 2.0, so the original license and its terms carry over.
Read the license itself before you use this model.