Qwen3.8-9B-Distill-Heretic-MTP
Decensored empero-ai/Qwen3.8-9B-Distill,
made with Heretic v1.4.0.
Safetensors version: petruhonk/Qwen3.8-9B-Distill-uncensored-heretic
The MTP head is kept. That is the point of this repository. The existing
abliteration of this model on the Hub ships 427 tensors and 32 layers — the
blk.32.nextn.* block is gone, and with it llama.cpp's --spec-type draft-mtp.
This conversion keeps all 442 tensors and 33 blocks, so speculative decoding
works: measured ~73 tok/s generation with it (draft acceptance 0.73), on an RTX 3060.
Performance
| Metric | This model | Original |
|---|---|---|
| Refusals | 6/100 | 100/100 |
| KL divergence | 0.0306 | 0 (by definition) |
Refusals are measured by Heretic on mlabonne/harmful_behaviors; lower is better.
Why another one
The published abliteration of this model
(rohit267/Qwen3.8-9B-heretic-uncensored)
reports 98/100 refusals against the original's 100/100 — two refusals removed, at
KL 0.0008. It barely changed the model, and it dropped the MTP head. This run
optimised further and preserved the head.
Usage
llama-server -m Qwen3.8-9B-Distill-Heretic-Q8_0.gguf \
-c 32768 --parallel 1 -ctk q8_0 -ctv q8_0 -fa on -ngl 99 \
--jinja --spec-type draft-mtp --spec-draft-n-max 3
--jinja matters: the chat template inside the GGUF carries tools and
<tool_call>, so function calling works and the model can drive an agent.
KV cache in q8_0 is not a quality choice but a memory one: f16 at 32k context
needs 4.12 GiB, which does not fit beside the weights on a 12 GiB card. With more
VRAM, drop to -ctk f16 -ctv f16.
The model is multimodal (Qwen3_5ForConditionalGeneration, vision + video); this
repository contains the text weights only. An mmproj file is needed for images.
Provenance
Abliteration and conversion done on a single RTX 3060 12GB: Heretic with
--quantization bnb_4bit, then convert_hf_to_gguf.py (without --no-mtp, so
the nextn block is carried over), then llama-quantize to Q8_0.