Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-KERN
Four measured mixed-precision GGUFs of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, selected by capability testing rather than KL divergence alone.
Quantized directly from DavidAU's BF16 Safetensors using a CF27-native imatrix (calibration-v6) and per-tensor type allocations derived from a multi-round sensitivity study against CF27's own BF16 logits. Not a requantization and not a blind transplant of the base Qwen3.8 allocation.
The tiers
| Tier | File GB | Capability | KLD | Recommended for |
|---|---|---|---|---|
| eqQ4_K_M | 14.73 | 95% of BF16 | 0.028 | Best fidelity |
| eqQ3_K_M | 13.04 | 91% of BF16 | 0.042 | High fidelity, fits comfortably on 16 GB |
| eqQ3_K_S | 11.59 | 81% of BF16 | 0.070 | Balanced |
| eqIQ2_M | 9.58 | 76% of BF16 | 0.160 | Smallest, but surprisingly capable |
The eq prefix means capability-equivalent to the stock recipe of that name.
eqQ4_K_M, for instance, matches the capability of a stock Radermacher Q4_K_M
while being 2.1 GB (12%) smaller.
eqIQ2_M is the exception. It is named for its size class, not its capability. On our probe it matches a stock Q3_K_S that is 2.7 GB larger, and outscores every published quant we tested that is smaller than that Q3_K_S.
Capability is scored on a private multi-sample probe as a percentage of what the BF16 model gets right. It drops sharply between these four rungs — intermediate quants that looked good on KLD alone did not survive the capability suite.
KLD is wikitext KL divergence from BF16 (200 × 512-token chunks, held-out).
Compared to published quants
For every published CF27 quant we tested, there is a smaller KERN tier that is at least as capable:
How to run
Each tier also comes as a -noMTP file with the MTP head removed (183 MB smaller).
Use it with DFlash 2 or without speculation, where the head is never used.
With MTP (embedded speculative head)
Every tier has an embedded MTP head (Q3_K, 183 MB, identical across all tiers).
Use draft length 2 (--spec-draft-n-max 2) for eqQ3_K_M, eqQ3_K_S and eqIQ2_M,
and 3 for eqQ4_K_M.
llama-server \
-m Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-KERN-eqQ3_K_M.gguf \
-ngl 99 -c 16384 \
-ctk q8_0 -ctv q8_0 \
--spec-type draft-mtp --spec-draft-n-max 2 \
--flash-attn on --jinja --parallel 1
eqQ4_K_M needs about 16 GB at 16k context, which is right at the edge of a
16 GB card. If the card also drives your desktop it will spill into system memory
and decode about a quarter slower; -ub 256 recovers most of the prompt speed but
not the decode. It is comfortable on 24 GB cards and unified memory. The other
three tiers fit a 16 GB card at the default -ub 512.
With DFlash 2 (external drafter)
All four tiers also work with an external DFlash 2 drafter. A compatible drafter is available at andrew-paul/Qwen3.8-27B-DFlash2-Q3_K_M-GGUF. DFlash 2 requires a compatible llama.cpp build.
llama-server \
-m Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-KERN-eqQ3_K_S-noMTP.gguf \
-md Qwen3.8-27B-DFlash2-Q3_K_M.gguf \
-ngl 99 -c 16384 \
-ctk q8_0 -ctv q8_0 \
--spec-type draft-dflash --spec-draft-n-max 4 \
--flash-attn on --jinja --parallel 1
Note that eqQ4_K_M does not fit alongside the external drafter on a 16 GB card.
Without speculation
All four files work without speculation. Speculative decoding is a throughput optimization, not a requirement.
What the tiers are made of
Every tensor's type is recorded in each GGUF's header, so a tier's full allocation can be read straight from the file. The MTP head (blk.64) is Q3_K across all tiers. Q4_K and Q8_0 heads measured no faster. A Q8_0 head can raise draft acceptance slightly (by up to about 3 points on the weakest trunk tested) but did not turn that into speed. Q2_K was the only precision cliff.
From the quantization study
-
KLD does not predict capability. A quant can be close on KLD and well behind on what it actually gets right. The capability probe found hard cliffs that KLD alone does not reveal.
-
Draft length 2 is fastest for the three smaller tiers; 3 for eqQ4_K_M. n=3 decodes 1.9–3.6% slower than n=2 on eqQ3_K_M, eqQ3_K_S and eqIQ2_M, and 1.9% faster on eqQ4_K_M.
-
KV cache at q8 costs ~0.002 KLD and frees ~0.5 GB at 16k context. KV q4 costs ~0.006 KLD and roughly doubles the context ceiling on memory- constrained cards.
Caveats
- Capability was measured on a private benchmark. It is an estimate, not a certified score.
- Vision (mmproj) was not tested as part of this study and may need separate validation.
- KV q4 KLD was measured at short context. Cache error may compound at longer contexts.
Community
eqIQ2_M has been confirmed running on an Adreno GPU (16 GB mobile LPDDR5 RAM) with a custom Turnip build at a couple tok/s.
Provenance and license
- Derivative model: DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, Apache-2.0
- Base model: Qwen/Qwen3.8-27B
- DFlash 2 drafter: andrew-paul/Qwen3.8-27B-DFlash2-Q3_K_M-GGUF
- Quantization/runtime: llama.cpp
- Calibration: imatrix computed on CF27 from Bartowski's Qwen3.8 calibration-v6 dataset
See SHA256SUMS for artifact hashes.