Quality-first GGUF quants of Qwen3.8-27B

Consider this fine-tune instead, it is even better: Kiwen1.2-27B

In the process of creating these quants:

• 95 hours of 2xGPU near 100% load were spent!
• 27 TB of not so good ggufs were materialized, measured and put to trash (all were usable)!
• 72 man-hours were spent!

These quants were produced by my quality-first experimental tool, niki-allocator. The main difference of my method is per-tensor type allocation (not per tensor groups, layers, community lore etc.) Each tensor gets what is estimated to be best for it. After that best allocations are tested with real KLD runs to choose the winner.

Source model

Quantization quality comparison

To prove my method I've performed comparison with quants from other providers:

unsloth, AtomicChat, bartowski, lmstudio-community, mudler, byteshape

Lower is better

Main metrics

Tail metrics

p99.9 are extremely corpus-driven with usual test corpus sizes of ~300-400K tokens

Same top probability

Useful for temperature 0, otherwise RMS Δp should be considered.

Note that graphs show 1 - same top p, so lower is better.

Perplexity (PPL)

Very crude noisy metric, cheaper to compute though, that's why so popular

Notes

TL;DR difference of corpora for end user: niki-small-plus-2 - coding, agentic, reasoning, hard logic prompts, English, Russian; independent-v2 - multilingual, English, web texts, science papers, math, legal, no reasoning or agentic traces, no code.

Unsloth 3.0 preview, AtomicChat, bartowski, lmstudio-community quants for comparison were downloaded on August 16, 2026. Unsloth 3.0, mudler, byteshape quants for comparison were downloaded on August 21, 2026. I've deliberately dropped some known bad quants (legacy quant types and such) to save my time on downloading and measuring.

Unsloth v3.0 was introduced after I finished my work. I added them for clean honest comparison. I don't like their marketing attitude and evidence presentation (just drop in their best graph without details on how exactly it was measured and if it is meaningful metric for end user). But after all they provided really good quants (mostly).

Unsloth v3.0 dropped MTP for lower quants reasoning that they are not needed. Well, I can't agree. From my findings MTP works well on lower quants. I added these no-MTP quants to comparison, but added fixed-size correction of their MTP v3.0 fixed-choice for all their non-MTP quants. Yeah, I know.. So for "honest" scientific comparison of only target models without strange choices of MTP quantization, here is comparison for all quants where only size of target model is accounted. This is not that honest for the end user though. End user downloads quant and either fits it in his VRAM/RAM or not, poor choice of oversized MTP can't be excuse here. Quant provider should provide MTP quantization based on their research, not just drop in something, or just delete it. It is part of a quant. But here you go:

All graphs with MTP size subtracted

Main metrics

Tail metrics

p99.9 are extremely corpus-driven with usual test corpus sizes of ~300-400K tokens

Same top probability

Useful for temperature 0, otherwise RMS Δp should be considered.

Note that graphs show 1 - same top p, so lower is better.

Perplexity (PPL)

Very crude noisy metric, cheaper to compute though, that's why so popular

See the tables below for exact numbers for all quants and metrics

Sizes

I've chosen 15 sizes to fill 8500-23000 MiB with geometric progression and rounded to 100 MiB

niki-allocator is able to produce allocation with ~1 MiB precision.

Which one to choose?

If you are after quality: the bigger - the better. So it is always a compromise of what you can afford considering context size and other constraints. Choose the biggest you can fit.

I haven't measured speed since it is highly hardware and configuration dependable. What is faster for me may be slower for you. If the speed is the only that matters and you have Blackwell - get some NVFP4 quant (but quality is considerably worse for the same quant size). Otherwise you should measure speed yourself on your hardware and your configuration to make real decision. By the way, on my setup ubatch is the parameter that has huge impact on speed. On 27B models 384 was found to be optimal in my case (2x5060Ti), 512 or 256 are much slower. So probably you should test this first.

Quant names

Hugging Face requires quant type in the filename to properly show it. But all my quants are mixed by design. I decided to include dominating quant type (by size, not by tensor count) to the name therefore. It may easily be less than 50% of the model though. So choose by size, ignore listed quant type. For my quants: bigger --> better quality. Always.

Methodology

niki-allocator

It is quality-first per-tensor quantization allocator for llama.cpp llama-quantize.

Allocator decides, per tensor, which quant type to use so that the resulting GGUF hits an exact target file size with the best estimated quality.

It is not direct measurement of KLD-metrics, it is only estimation. But as you can see from real KLD evaluations estimation is good enough.

Raw results are already good. But I added some calibration levers on top to make it better (for Qwen3.5-0.8B I used uncalibrated estimation). These levers need series of KLD evaluations to move estimation closer to reality, so it takes hours-days of GPU time on my consumer setup.

On top of this I added bayesian search seeded with best results from levers to move calibration even further. This is very GPU-intensive, literally took days (nights actually) of GPU time on my setup.

Corpora

• imatrix: niki-small-2 corpus - ~330K tokens from my own agentic sessions (English, Russian, design docs with hard logic, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns. Turn-aligned by tokenizer so that each chunk contains exactly 1 turn. Short turns dropped. Used for imatrix only.
• Calibration of niki-allocator: hardest 16 chunks of niki-small-plus-2 for faster crude KLD calibration (full corpus is too slow on my hardware).
• Evaluation 1: niki-small-plus-2 corpus - ~400K tokens from my own agentic sessions (English, Russian, design docs with hard logic, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns. Turn-aligned by tokenizer so that each chunk contains exactly 1 turn. Short turns dropped. Does not include any exact or similar (by simhash) agentic turns from niki-small. So while it is from the same domain as imatrix-corpus, it is different.
• Evaluation 2 (truly independent this time): independent-v2 corpus - ~390 K tokens. This corpus is random compilation of top-13 languages prose (HuggingFaceFW/fineweb-2) excluding English (it is everywhere in other domains already), legal domain (common-pile/caselaw_access_project), math domain (open-web-math/open-web-math), science domain (recursal/arXiv-CC0-v0.5). Code and agentic data wasn't included since my quants are highly tuned for these already (it wouldn't be honest comparison). This dataset wasn't seen during calibration or imatrix at all. Used only for final evaluation before publishing. So my quants are not anyhow tuned for this dataset. Foreign quants haven't seen this exact dataset as well (random pick).

Notes:

• independent-v2 corpus will be published here. Note that you can't redistribute it without proper attributions due to license of some included papers.
• niki-* corpora are not published. There is nothing really special actually. They just contain my private information. Sorry for inconvenience.
• Why not wikitext-2 test? While wiki may be considered as independent evaluation corpus for my quants. It easily may not be true for others quants (wikitext-2 is the most popular corpus somehow for such jobs). They may have seen it (or part of it) during imatrix creation or their methodology calibration. So for really independent evaluation there should be corpus that wasn't seen anyhow by any quants and even better if it wasn't seen during base model creation. Also wiki is mostly homogenous (not very easy, but there are very few hard tokens and chunks). My corpus is everything from "walk in the park" to "high mountain climbing". Its tail metrics are much higher. It is actually more representative for the real model work (if crawling wiki is not your main use case of course). So I use my own corpus because I need these quants to be good in my tasks. And independent-v2 as truly independent evaluation.

Chunk sizes

• imatrix (niki-small-2): 8064 tokens (384 x 21). 41 chunks total. To catch reasoning and long turn traces, not just basic chat.
• niki-small-plus-2 evaluation: 6144 tokens (384 x 16). 65 chunks total. To catch reasoning and long turns, but to keep chunks amount somewhat reasonable (that's why lower than for imatrix).
• independent-v2 evaluation: 2304 tokens (384 x 6). 171 chunks total. Long enough for chat and basic tasks.

Why 384 magic number?

My hardware is fastest with ubatch 384, so I use it.

Metrics

All metrics were evaluated with llama-perplexity against Q8_0 standard quant (due to hardware constraints).

Quantization was performed with patched llama-quantize from BF_16 source. Fully compatible with original llama.cpp.

All my and foreign quants were evaluated against the same base.

llama.cpp commit 876a4321163249c43ca4e986818fab5ab081f282.

The main metrics during calibration of niki-allocator were mean KLD and RMS Δp.

Better PPL doesn't always mean better other more important metrics. So it can't be used as a main metric (some people still use). It is just fast indicator (no need for logits base) to check for big problems. It can't be main quality metric. Don't make or accept comparisons done only by PPL. It is noise.

Notes on p99 KLD, p99.9 KLD, max KLD.

With such a small corpora these metrics are not very robust. While p99 KLD can give some plausible comparison for the allocation method as a whole, p99.9 is extremely corpus-driven (too little data), max KLD is just random at any corpus size (one hardest token of the whole dataset, saying nothing about other hard tokens). So I don't tune to these. And anybody who does with less than several M tokens corpus is just over-fitting to their exact corpus. Please correct me if I am wrong.

Evaluation results

niki-small-plus-2 corpus

Lower is better except same top p

independent-v2 corpus

Lower is better except same top p

MTP

I've tested MTP performance and discovered that there is no point to keep MTP head higher than IQ4_XS. Q3_K is near but slightly worse (used for lower quants). Exact allocation of IQ4_XS/Q3_K is chosen by allocator (not pinned). Lower than that there is considerable quality drop, so not used in my quants. Higher is just waste of model size and VRAM.

For numbers check my previous GRM-2.6-Plus-0628 quant's MTP section.

I've done fast tests on other models (same architecture). Same conclusions.

Also I've additionally checked fast if there is any degradation on IQ4_XS vs Q8_0 with longer context (tested up to 28K), none found. And temperature = 0.6, still no evidence of degradation. But I must say that doing this test properly and scientifically much time and effort is needed. Since not 0 temperature gives different results: many runs needed to stabilize. Reasonably long context (100K+ from my opinion) is very slow on my hardware. So there actually may be some degradation. But that should be proven and not taken by "it seems" or "everybody knows". If there is such comprehensive analysis already, please let me know.

Basically I use these parameters for MTP: --spec-draft-n-max 5 --spec-draft-p-min 0.8. Primary reason of poor results with MTP is ignoring --spec-draft-p-min. Don't put obvious crap in your drafts. But let drafter do more if it is sure enough. MTP performance speed-up is highly dependable on hardware, configuration and specific context. So to decide what is best for your case you should test yourself.

Also don't use other llama.cpp drafters like ngram-mod with MTP. MTP wastes time even if ngram-mod already done draft. That is how llama.cpp is programmed. Not obvious. I have created experimental fork to overcome this issue (and added much better ngram-mod-v2 + minor fixes): https://github.com/NikiKrutan/niki-llama.cpp. I use it myself on a daily basis, but it is more like crude draft than real working fork. It breaks some llama.cpp behavior. But it gives another ~1.5x speed-up on top of MTP for my use cases.

Chat template

Chat template included with my gguf's is v22.2 from froggeric. There may be newer version as it is often updated.

If you have any problems with it you may also try one from Moore2877.

Multimodality

I've quantized mmproj following DAXZEIT's receipt for Qwen3.6 that I used in Qwen3.6 era.

It reduces mmproj size saving practically same quality. Why reducing size? Well, it is probably not needed for most use cases. But with this I can put my 17300 quant with q8_0 K-cache and q4_0 V-cache + MTP with the same KV-cache quantization + mmproj + 262K full context in my 2x16Gb VRAM. With BF16 mmproj I can't do it despite very little difference in size. Every MiB matters when you are about quality.

Also any standard Qwen3.8-27B mmproj should work if you want full BF16 one.

Acknowledgments

• Qwen - great base model. Really great!
• llama.cpp - quantization and evaluation framework.
• unsloth - quants for comparison.
• AtomicChat - quants for comparison.
• bartowski - quants for comparison.
• lmstudio-community - quants for comparison.
• mudler - quants for comparison.
• byteshape - quants for comparison.
• FineWeb - data for evaluation dataset.
• caselaw_access_project - data for evaluation dataset.
• open-web-math - data for evaluation dataset.
• recursal - data for evaluation dataset.
• DAXZEIT - mmproj receipt.
• michaelw9999 - some ideas to start with.
• Required attributions for exact science papers used for evaluation dataset are on dataset page.