Qwopus3.8-27B-Flash ATX-IQ4_XS-M (GGUF)
The ATX-IQ4_XS-M recipe applied to Jackrong/Qwopus3.8-27B-Flash, a fine-tune of Qwen3.8-27B tuned for shorter reasoning and higher MTP acceptance in agent loops. Same architecture, same 27.32B parameters, same 866 tensors as the base model, so the recipe carries over unchanged; what is new here is a model-specific importance matrix.
The recipe was designed for a single RTX 3090 / 3090 Ti (24 GB): fit a populated 200K-token context with an 8-bit key cache and the model's own MTP speculative head, and decode faster per speculative round than the stock Q4 mixes on that card. The original build and its measurements are at jakeatx/Qwen3.8-27B-ATX-IQ4_XS-M-GGUF.
| file | size | notes |
|---|---|---|
Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf | 14.6 GiB (15,724,533,952 bytes) | 4.60 bits per weight, MTP layer included |
imatrix_qwopus_flash.gguf | 13.6 MB | importance matrix computed from this model, see below |
tensor_types_ATX-IQ4_XS-M.txt | the per-tensor type map used by llama-quantize |
On the name. ATX-IQ4_XS-M reads as: base format IQ4_XS on the bulk tensors, upgrade pattern M. In llama.cpp's vocabulary the XS in IQ4_XS is part of the format's name (the 4.25 bits-per-weight super-block layout of the IQ4 codebook), not a mix size, while the S/M/L suffix on the K-quants describes how many tensors are lifted above the base format. This file lifts the same tensors Q4_K_M does, plus the early-layer tensors Jackrong's Q4_K_S lifts, so it is an M-pattern mix on an IQ4_XS base: 4.60 bits per weight, between Q4_K_S (4.63 for this model) and Q4_K_M (4.92) in size.
Recipe
Built from the Qwopus safetensors converted to a BF16 GGUF with llama.cpp's converter (MTP layer kept), quantized with the per-tensor type map:
| tensors | format | share of weight bytes |
|---|---|---|
| everything not listed below | IQ4_XS | ~68% |
| attn_output, ssm_out, ffn_down in the layers Unsloth's tier ladder upgrades first | Q5_0 | ~17% |
| attention K/V projections, output head | Q6_K | ~7% |
| token embedding (host side) | Q4_K | ~5% |
| the eight attention K/V tensors Q4_K_M keeps at Q8_0 (V in layers 11, 27, 31, 51, 55, 59, 63; K in 31) | Q8_0 | |
| FFN down in layers 0-7 and GDN qkv in layers 0-2 (the tensors Jackrong's Q4_K_S lifts to Q5_K) | Q5_0 | ~3% |
| MTP draft layer (blk.64) | Q5_0 | |
| GDN alpha / beta vectors (96 tiny) | Q8_0 |
llama-quantize --imatrix imatrix_qwopus_flash.gguf --tensor-type-file tensor_types_ATX-IQ4_XS-M.txt \
--token-embedding-type q4_K Qwopus3.8-27B-Flash-BF16.gguf Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf iq4_xs
Every tensor's type was checked against the base ATX-IQ4_XS-M file after quantization: identical except the eleven early-layer tensors above, which this build carries at Q5_0 so that every tensor Jackrong's own mix protects stays protected here.
Importance matrix. Rather than reuse the Unsloth base-model imatrix, this build computes its own from Qwopus at Q8_0 precision over about 226K tokens: Bartowski's calibration_datav3 (mixed prose, code, multilingual, chat) plus 64K-token agentic and coding prompts and a 32K-token retrieval-analysis prompt from the production prompt set the recipe was tuned on. A fine-tune shifts activation statistics; measuring them on the model being quantized is the more faithful translation of the recipe.
Why this mix: on SM86 the fastest weight format per tensor at speculative verification widths 1-5 is IQ4_XS, not the 2-3 bit codebook types (IQ3_S, IQ3_XXS, IQ2_S are instruction-bound and slower despite fewer bytes); Q5_0 is about 16% cheaper than Q5_K; Q8_0 is the only format near the memory roof. Extra bits go where Unsloth's tier ladder puts them (attention V/K, attention output, GDN output, FFN down).
Measured (RTX 3090 Ti, TurboQuant+ fork with SM86 kernel work, MTP-3, q8_0 K / turbo3 V)
Same prompts for both files, sampled decode at temperature 1, top-k 20, top-p 0.95, 256-token generations, two runs per cell (second run with the prompt cached). Compared against Jackrong's Q4_K_S, the like-for-like size class (14.7 GiB, 4.63 bpw).
| context | workload | ATX-IQ4_XS-M decode tok/s | Q4_K_S decode tok/s | ratio | prefill tok/s (ATX / Q4_K_S) |
|---|---|---|---|---|---|
| 16K | coding | 71.9 | 59.7 | 1.20 | 1378 / 1298 |
| 16K | agentic | 78.8 | 70.3 | 1.12 | 1393 / 1388 |
| 16K | RAG analysis | 83.8 | 67.4 | 1.24 | 1385 / 1377 |
| 32K | coding | 72.7 | 64.8 | 1.12 | 1290 / 1288 |
| 32K | agentic | 77.1 | 65.4 | 1.18 | 1286 / 1283 |
| 32K | RAG analysis | 76.7 | 60.2 | 1.27 | 1285 / 1281 |
| 64K | coding | 66.2 | 55.1 | 1.20 | 1099 / 1099 |
| 64K | agentic | 68.8 | 53.0 | 1.30 | 1095 / 1103 |
| 64K | RAG analysis | 67.1 | 57.9 | 1.16 | 1095 / 1104 |
| 100K | coding | 56.2 | 57.1 | 0.98 | 934 / 940 |
| 100K | agentic | 57.9 | 55.7 | 1.04 | 933 / 940 |
| 100K | RAG analysis | 52.6 | 48.3 | 1.09 | 931 / 938 |
| 16K mean | all three | 78.1 | 65.8 | 1.19 | |
| 32K mean | all three | 75.5 | 63.5 | 1.19 | |
| 64K mean | all three | 67.4 | 55.3 | 1.22 | |
| 100K mean | all three | 55.6 | 53.7 | 1.04 |
Decode is the mean of the two runs per cell; the ratio is ATX-IQ4_XS-M over Q4_K_S. Ready VRAM at a 110,592-token window: about 19,910 MiB (ATX-IQ4_XS-M) against 19,879 MiB (Q4_K_S). The two files prefill at the same rate. Sampled decode swings with draft acceptance (65-80% per cell here), and a few generations stopped early on their own (the model finished its answer before 256 tokens), which makes the 100K cells the noisiest: read the picture as roughly +20% decode from 16K to 64K, narrowing to a few percent at 100K, where attention over the long cache takes a growing share of each round and the weight format matters less. Same runtime, same prompts, same session for both files; Jackrong's Q4_K_S was measured as downloaded.
sha256 of the GGUF: 733198976d76de46326f9d685533827551d14cdef46f03c2a4d6255f42df76a2.
Run it
Runtime with the kernel and memory work: https://github.com/JakeATX/llama-cpp-qwen-ampere, branch main. Build with -DGGML_CUDA=ON -DGGML_CUDA_FA=ON -DCMAKE_CUDA_ARCHITECTURES=86. The file also loads on stock TurboQuant+ and on mainline llama.cpp (without the turbo3 cache there), with less context headroom.
GGML_Q8_TURBO3_MMA_FUSED=1 llama-server -m Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf \
-c 245760 -b 4096 -ub 1024 -t 8 -tb 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
--parallel 1 --jinja --fit off \
--cache-prompt --cache-ram 8192 --ctx-checkpoints 24 --checkpoint-min-step 10240 \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.45 \
--spec-draft-type-k q8_0 --spec-draft-type-v turbo3
Single-user configuration (one request at a time). The cache flags are explained on the base model's card; they keep long conversations from re-prefilling and let you edit or regenerate a turn on this partly recurrent architecture. Qwopus ships its own chat template and sampling defaults (temperature 1, top-p 0.95, top-k 20), which the GGUF carries.
Quality was not re-measured for this build; the recipe's quality evidence is on the base model's card and in its paired Terminal-Bench / SciCode comparison against Q4_K_M. Qwopus's own capability trade-off (shorter reasoning, lower MMLU-Pro than the base) is described on Jackrong's card.
Credits
Qwen team for Qwen3.8; Jackrong for Qwopus3.8-27B-Flash; Unsloth for the dynamic-quant tier ladder this recipe follows; Bartowski for the calibration set; TheTom for TurboQuant+.
Qwen3.8 Perian project lineage
This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Baseline/comparator quant retained with the project checkpoints. It is not a Perian-pruned or Perian-QLoRA artifact.
The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:
- depth: 48 to 32 transformer layers;
- routed-expert width: 384 to 288 experts per layer;
- PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
- rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.
Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.