Qwen3.8-27B Heretic ARA MXFP6/MXFP8 with DFlash2
This repository contains a custom GGUF target and its companion DFlash2 draft for Blackwell-optimized llama.cpp inference.
Use the matching runtime: sirus20x6/llama.cpp-blackwell-dflash2.
Files
| File | Purpose | Size | SHA-256 |
|---|---|---|---|
Qwen3.8-27B-heretic-ara-MXFP6-MXFP8.gguf | 27B target | 20.72 GiB | 238141420ea8f5e8f051dd5c42b741ad09d02565f540aba5ac233dff7f0490c1 |
Qwen3.8-27B-DFlash2-MXFP6_E2M3.gguf | DFlash2 companion draft | 1.47 GiB | 590f4b562f23019f28871bad8c925d6932ccde79c88e27c3228e7d9159859620 |
The target has 27,320,697,856 parameters and is approximately 6.51 BPW. Its matrix tensor mix is:
- 396 MXFP6_E2M3 tensors;
- 110 OCP MXFP8 tensors;
- MTP tensors preserved.
MXFP6_E2M3 is an experimental GGUF extension and is not compatible with stock llama.cpp. Use the linked fork.
Sources
- Target source:
heretic-org/Qwen3.8-27B-heretic-ara - Base model:
Qwen/Qwen3.8-27B - Draft source:
incoai/Qwen3.8-27B-DFlash2 - Quantization/runtime source:
sirus20x6/llama.cpp-blackwell-dflash2 - Advanced quantizer:
michaelw9999/advanced-gguf-quantizer
Download and run
git clone https://github.com/sirus20x6/llama.cpp-blackwell-dflash2
cd llama.cpp-blackwell-dflash2
./build-blackwell-dflash2.sh
hf download \
sirus/Qwen3.8-27B-heretic-ara-MXFP6-MXFP8-DFlash2-GGUF \
Qwen3.8-27B-heretic-ara-MXFP6-MXFP8.gguf \
Qwen3.8-27B-DFlash2-MXFP6_E2M3.gguf \
--local-dir models
./run-qwen3.8-heretic-dflash2.sh
The launcher defaults to one 262144-token slot, MXFP8 K/V cache, and seven DFlash2 draft tokens. The checkpoint's trained block size is eight, including one anchor token.
Performance note
The earlier tri-quant target was slower in our observed testing. A previous version of this card included throughput figures collected with differing runtime, cache, context, and execution settings. Those values were not a valid controlled comparison and have been withdrawn. No comparative performance claim is made here until the artifacts are rerun under an identical benchmark configuration.
Validation and limitations
- Structural GGUF inspection passed.
- MTP preservation passed.
- Completion smoke tests passed.
- Device-resident DFlash2 handoff was tested through a 65,588-token request.
- No final PPL/KLD evaluation was produced, so this release makes no quantified quality claim against BF16.
- The target artifact is text-only.
- MXFP6_E2M3 and the bundled runtime are experimental.
License and attribution
The source target and DFlash2 draft are Apache-2.0. See LICENSE.
The runtime repository retains llama.cpp, advanced-gguf-quantizer, TurboQuant, and Four Over Six notices and links. DFlash2 originates from Inco AI. The Heretic ARA model originates from heretic-org and the Heretic project.