Qwen3.8-27B GGUF for Strix Halo
This repository contains the target and DFlash2 GGUFs selected during a ROCm optimization study on AMD Strix Halo (gfx1151). The files are ordinary GGUF models, but they were quantized and validated together with a llama.cpp branch containing the UMA scheduler ring and the ROCm TOP_K implementation.
Files
| File | Size | Contents |
|---|---|---|
Qwen3.8-27B-IQ4_XS-ALL-IMATRIX-Q8-OUT-MTP.gguf | 16,110,851,680 bytes | Main model: 496 IQ4_XS, 10 Q8_0, and 360 F32 tensors |
Qwen3.8-27B-DFlash2-IQ4_XS.gguf | 1,038,313,376 bytes | DFlash2 draft: 44 IQ4_XS, 5 Q5_K, and 32 F32 tensors |
The compatible Q8_0 multimodal projector is not duplicated in this repository. Supply mmproj-Qwen3.8-27B-Q8_0.gguf separately when using image input.
Main-model quantization
The main GGUF was quantized from the Bartowski BF16 GGUF, including its embedded MTP layer, using the published Bartowski importance matrix.
- All 496 active matrices covered by the importance matrix use IQ4_XS.
output.weight,token_embd.weight, and the eight block-64/MTP matrices remain Q8_0 because the published matrix does not calibrate them.- One-dimensional and convolutional structural tensors remain F32.
- An explicit tensor map avoids the Q5_K substitutions made by the ordinary
MOSTLY_IQ4_XSpreset.
On 16 held-out 512-token chunks, the measured perplexity was 15.3977 +/- 0.73357, compared with 15.1721 +/- 0.72292 for the source BF16 GGUF. This is a small local evaluation, not a general model-quality benchmark.
DFlash2 quantization
The DFlash2 source is z-lab/Qwen3.8-27B-DFlash2. The IQ4_XS draft was requantized from its Q8_0 GGUF.
Standalone DFlash importance-matrix collection currently requires the target context and cannot be run directly through llama-imatrix. A transferred target-model matrix was therefore used:
- 18 draft tensors received exact target-name and input-width matches.
- 23 received the mean importance vector for target tensors with the same input width.
- 8 received uniform weights where no compatible target vector was available.
The standard IQ4_XS preset retained the five attention-V matrices as Q5_K. The resulting draft is 49.51% smaller than the 2,056,414,816-byte Q8_0 source.
Strix Halo results
The counterbalanced server test used the same target model, custom HIP/ROCr retained-PM4 runtime, deterministic 128-token workloads, two independently loaded servers per configuration, and fixed DFlash2 widths 3 and 6.
Maximum draft width 3
| Workload | Q8_0 draft | IQ4_XS draft | Change | Q8 acceptance | IQ4 acceptance |
|---|---|---|---|---|---|
| Prose | 25.257 t/s | 25.742 t/s | +1.92% | 48.70% | 47.13% |
| Reasoning | 30.739 t/s | 32.481 t/s | +5.67% | 69.11% | 71.67% |
| JSON | 39.587 t/s | 41.032 t/s | +3.65% | 98.96% | 98.96% |
Maximum draft width 6
| Workload | Q8_0 draft | IQ4_XS draft | Change | Q8 acceptance | IQ4 acceptance |
|---|---|---|---|---|---|
| Prose | 25.215 t/s | 25.666 t/s | +1.79% | 30.34% | 29.30% |
| Reasoning | 37.797 t/s | 39.143 t/s | +3.56% | 57.65% | 57.65% |
| JSON | 56.483 t/s | 58.529 t/s | +3.62% | 95.58% | 95.58% |
The deterministic target-output hash matched between the Q8_0 and IQ4_XS drafts for every workload. These figures characterize the tested prompts and runtime; speculative throughput depends strongly on prompt-dependent acceptance.
Without speculative decoding, the main model measured 14.0976 t/s for tg128 with retained PM4. The retained command-list path improved that result by about 3.2% over ordinary AQL in the matched test.
Runtime
The tested software branches are:
Example llama-server invocation:
export LD_LIBRARY_PATH=/path/to/custom/hip/lib:/path/to/custom/rocr/lib:/opt/rocm/core-10.0/lib:/opt/rocm/core-10.0/lib/llvm/lib
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1
export ENABLE_RETAINED_PM4=1
export DEBUG_HIP_GRAPH_PM4=1
llama-server \
-m Qwen3.8-27B-IQ4_XS-ALL-IMATRIX-Q8-OUT-MTP.gguf \
--mmproj /path/to/mmproj-Qwen3.8-27B-Q8_0.gguf \
--mmproj-device ROCm0 \
-dev ROCm0 \
-ngl 999 \
-fa on \
-fit off \
--load-mode none \
-c 65536 \
-b 2048 \
-ub 512 \
--parallel 1 \
--jinja \
--spec-type draft-dflash \
--spec-draft-model Qwen3.8-27B-DFlash2-IQ4_XS.gguf \
--spec-draft-device ROCm0 \
--spec-draft-ngl 99 \
--spec-draft-n-min 0 \
--spec-draft-n-max 6 \
--spec-draft-p-min 0.10
Width 3 was marginally faster for the low-acceptance prose test. Width 6 was substantially faster for the higher-acceptance reasoning and structured-output tests.
Checksums
9e5f86c794b45b215a2768723c94819500c0c8f896094d628728e6fc94cd6324 Qwen3.8-27B-IQ4_XS-ALL-IMATRIX-Q8-OUT-MTP.gguf
11c7848014bd68040a42837b381bbefff5d0acc22cf20b6055a48d560c834445 Qwen3.8-27B-DFlash2-IQ4_XS.gguf
License
The source model and DFlash2 metadata declare the Apache 2.0 license. Users remain responsible for reviewing the upstream model repositories and their terms.