Qwen3.8-27B - Abliterated (GGUF, imatrix)
Imatrix-quantized GGUF ladder of RobinsonLabs/Qwen3.8-27B-abliterated, which is an abliterated bf16 base of Qwen/Qwen3.8-27B.
The MTP head is abliterated in-band and the vision tower is preserved -- see the base repo for the method, the verification, and the measured refusal numbers.
fp16 base
The full-precision master these were cut from is RobinsonLabs/Qwen3.8-27B-abliterated -- bf16 safetensors, carrying the method, the verification, and the measured refusal numbers. Go there to re-abliterate, LoRA-merge, fine-tune, or roll your own quants.
Vision
mmproj-Qwen3.8-27B-abliterated-f16.gguf (334 tensors) is published here. Download it
alongside whichever main quant you pick and pass it with --mmproj to get the vision half.
Without it you have a capable text model and no image input.
llama-server -m Qwen3.8-27B-abliterated-Q4_K_M.gguf \
--mmproj mmproj-Qwen3.8-27B-abliterated-f16.gguf \
-ngl 99 -c 32768
Quants
| file | bits | size | bpw | fits |
|---|---|---|---|---|
Q8_0 | 8 | 29.05 GB | 8.51 | 2x24GB, or 32GB+ |
Q6_K | 6 | 22.43 GB | 6.57 | 24GB card, quality ceiling |
Q5_K_M | 5 | 19.54 GB | 5.72 | 24GB comfortable |
Q4_K_M | 4 | 16.84 GB | 4.93 | 24GB / 16GB with offload -- the volume rung |
IQ4_XS | 4 | 15.37 GB | 4.50 | 16GB card |
Q3_K_M | 3 | 13.59 GB | 3.98 | 16GB tight |
IQ3_XS | 3 | 12.26 GB | 3.59 | 12GB card |
IQ2_M | 2 | 10.30 GB | 3.02 | 10-12GB card -- quality-compromised, read the note |
Sizes are the built artifacts, exact. bpw is bytes x 8 over the model's own 27,320,697,856
parameters, summed from the master's tensor shapes rather than taken off the "27B" in the name.
Every K/IQ rung is imatrix-guided except for the MTP block, which no imatrix covers -- see below
for what we do about it. Q8_0 uses no imatrix by design; it gains essentially nothing from
importance weighting.
What the low rungs actually cost
Measured rather than asserted. Perplexity over a held-out slice of wikitext-2 -- deliberately NOT
the operator corpus the imatrix was calibrated on, so this is an out-of-distribution read and not a
flattering one. What matters is each rung's distance from the Q8_0 reference on identical text.
| rung | PPL | vs Q8_0 |
|---|---|---|
Q8_0 (reference) | 5.9283 | -- |
IQ3_XS | 6.2212 | +4.9% |
IQ2_M | 6.7226 | +13.4% |
IQ2_M is a real quality step down and is labelled as such. It stays coherent -- it holds an
argument, follows a format instruction, and reasons correctly about physics in spot checks -- but
if you have the VRAM for IQ3_XS or above, take it. Ship IQ2_M when 10-12 GB is the constraint,
not because it is close to the top of the ladder.
We also built IQ2_XS and did not publish it. It measured 7.4030, or +24.9% against the
reference, to save 0.91 GB over IQ2_M -- roughly three times worse per gigabyte than the step
above it, and past the point where we are willing to put our name on the output. The file exists;
it is not here on purpose.
The imatrix is not the usual one
Most published imatrix quants calibrate on calibration_datav3.txt or similar generic English.
This ladder is calibrated on an in-domain corpus -- 1.8 MB / 17.5K lines of real technical
operator transcripts (infrastructure work, debugging, model-building dialogue), 120 chunks,
final PPL 10.3431 +/- 0.168.
That is a deliberate trade, not an accident. It biases the quantization error toward preserving behaviour on long technical dialogue, tool use, and operator-style instruction-following. If your use case is closer to general chat or non-English, a generic-calibrated ladder may serve you better and that is fine -- we would rather tell you the calibration than let you assume it.
What the imatrix does not cover
Being straight about a limitation, since the calibration is the selling point.
llama-imatrix collects its statistics during a perplexity-style forward pass, and that pass never
runs the model's MTP (multi-token-prediction) draft head. So the imatrix carries entries for
blk.0 through blk.63 -- the 64 trunk layers -- and nothing for blk.64, the MTP block.
llama.cpp handles that two ways, and only one of them tells you: above its "very low-bit"
threshold it quantizes the block blind at the trunk's depth and says nothing, and at IQ3_XS it
refuses and aborts the build. The loud case is the honest one, and it is how we found the quiet
one.
Rather than ship a block with no importance data at low precision, this ladder holds an invariant:
blk.64is never belowq5_K, and never an I-quant, in any rung.
Q8_0, Q6_K and Q5_K_M already satisfied that from the stock mixture and are untouched.
Q4_K_M, IQ4_XS, Q3_K_M and IQ3_XS pin it explicitly with
--tensor-type 'blk\.64\.=q5_K'. IQ4_XS is the one worth calling out: its MTP block would
otherwise be a blind iq4_xs, and I-quants are precisely the family that leans on importance
data. The pin costs roughly 0.1 GB per rung.
If you re-quantize this model yourself, check your imatrix's block coverage against n_layers
before you start, rather than discovering it at the bottom rung.
Disclosure
Abliterated: the hard-refusal reflex on adult / creative content is reduced via single-direction
weight orthogonalization. Harm guardrails are retained by design -- self-harm prompts still
redirect to help (e.g. 988) rather than comply. Not a jailbreak-for-anything model, not intended
to assist genuine wrongdoing. Tagged not-for-all-audiences. See the base repo for the full
disclosure and the measured base-vs-abliterated numbers.
Provenance
Built by Robinson Labs with ModelForge.
Base pinned at 1d4bf0f2. Quantized from our own bf16 abliterated master, not from someone
else's quant -- so the ladder is a single lineage, not a requant chain.