Qwen3.8-27B-Heretic-Dynamic-GGUF

Dynamic (Unsloth-recipe) GGUF quants of the Heretic-abliterated Qwen3.8-27B.

Currently available: UD-Q4_K_XL, UD-Q5_K_M, UD-Q6_K_XL, UD-Q8_K_XL. More sizes may be added if there is interest — each is built the same way described below (Unsloth per-tensor layout + imatrix on the Heretic BF16).

These quants combine two things that were previously only available separately:

• the Heretic MPOA abliteration from llmfan46/Qwen3.8-27B-Ultra-Uncensored-Heretic-Native-MTP-Preserved (uncensored, MTP-preserved), and
• Unsloth's Dynamic v3.0 per-tensor quantization layout + importance matrix, which the source repo's own quants (plain static K-quants) do not use.

The result is measurably closer to the full-precision model than a static K-quant of the same weights — verified by KL divergence at Q5 (below) — at a slightly smaller file size.

What this is, precisely

• Weights: llmfan46's Heretic BF16 (Heretic v2.0.0.dev0, a variant of Magnitude-Preserving Orthogonal Ablation). All 15 MTP tensors preserved; vision projector untouched.
• Quantization: llama-quantize --allow-requantize with Unsloth's published imatrix_unsloth.gguf and an exhaustive per-tensor --tensor-type-file generated from the corresponding Unsloth GGUF of the original model. Each size's tensor-type layout is verified identical to its Unsloth counterpart (866/866 tensors, 0 type mismatches).

Tensor-type distributions (F32 norms etc. excluded):

Why dynamic over the static Q5 — measured

KL divergence vs a non-abliterated Unsloth Q8_0 reference (wikitext-2 test, -c 8192, f16 KV; self-control floor: mean KLD 0.000000, top-1 99.994%). Because both Q5s carry the identical abliteration, the gap between them isolates the quantization method:

Dynamic is lower divergence on every axis — mean, median, tail, top-1 agreement — in a smaller file. (For reference, Unsloth's dynamic Q5 of the original non-abliterated model scores 0.021978 mean KLD / 96.933 % top-1 against the same base; the remainder of this model's divergence is the intended abliteration, not quantization loss.)

Files

All main GGUFs contain the 15 MTP tensors.

Qwen3.8-27B is a native vision-language model; the vision encoder lives in the separate mmproj-BF16.gguf. It is optional — text-only use needs just the main GGUF.

Usage (llama.cpp)

# text only
llama-cli -m Qwen3.8-27B-Heretic-Dynamic-UD-Q5_K_M.gguf -c 8192

# with vision (image/video input)
llama-mtmd-cli -m Qwen3.8-27B-Heretic-Dynamic-UD-Q5_K_M.gguf --mmproj mmproj-BF16.gguf -c 8192

Sampling (from Qwen3.8's own recommendation, thinking mode): temperature 1.0, top_p 0.95, top_k 20, min_p 0.0.

Reproduce

# per-tensor layout copied from Unsloth's GGUF of the same size, imatrix from Unsloth's repo
# (fallback base type: Q4_K_M / Q5_K_M / Q6_K / Q8_0 to match the size being built)
# 1. make the per-size --tensor-type-file from the combined map (here: UD-Q5_K_M)
awk -F'\t' 'NR==1 { for (i=2; i<=NF; i++) if ($i=="UD-Q5_K_M") c=i; next }
            { n=$1; gsub(/\./,"\\\\.",n); print "^" n "$=" $c }' \
  tensor_types.tsv > tensor_types_UD-Q5_K_M.txt

# 2. quantize
llama-quantize --allow-requantize \
  --imatrix imatrix_unsloth.gguf \
  --tensor-type-file tensor_types_UD-Q5_K_M.txt \
  Qwen3.8-27B-Ultra-Uncensored-Heretic-Native-MTP-Preserved-BF16.gguf \
  Qwen3.8-27B-Heretic-Dynamic-UD-Q5_K_M.gguf Q5_K_M

tensor_types.tsv (in this repo) holds the per-tensor type for every non-F32 tensor in every size, read from the GGUF headers of the corresponding Unsloth files — one row per tensor, one column per size. The generated regexes must be anchored (^…$) — llama-quantize matches them unanchored, and a bare output\.weight line would silently clobber every attn_output.weight.

Note: the imatrix was calibrated on the original (non-abliterated) model. Given how close the abliterated model is to the original, it remains a valid importance proxy — the same assumption Unsloth's and other abliterated dynamic quants rely on.

Credits

• Original model: Qwen/Qwen3.8-27B (Apache-2.0)
• Abliteration + BF16 source: llmfan46/Qwen3.8-27B-Ultra-Uncensored-Heretic-Native-MTP-Preserved, made with Heretic
• Dynamic quant recipe + imatrix: unsloth/Qwen3.8-27B-GGUF

Usage warnings

This is an uncensored model: its safety filtering has been substantially removed and it can produce sensitive, controversial, or otherwise inappropriate content. Review outputs, use it in controlled settings, and ensure your use complies with applicable law and the Apache-2.0 license of the base model. Responsibility for generated content rests with the user.