Qwen3-8B-Heretic
A abliterated ("heretic") derivative of the official Qwen/Qwen3-8B model. It is a
local, non-official variant produced by applying the Heretic abliteration protocol to
remove the base model's safety-aligned refusal behaviour while retaining general
capability.
Warning. This model has had its refusal behaviour deliberately reduced. It is intended for research and personal experimentation only. Do not deploy it in production systems that require content moderation or safety guarantees.
What Was Done
- Ablation: The Heretic protocol was applied (
qwen3-8b-heretic-200t, trial 131 of 200 sensitivity-aware trials) to abliterate the refusal direction in the base model. - Base model:
Qwen/Qwen3-8B(Apache-2.0, commitb968826d). - Parameters: ~8.2B (4 safetensors shards, ~15.3 GB total in
model_files/).
Capability Retention
Evaluated against the source model on 20 harmless prompts (seed=42, temperature=0.6, top_p=0.95, top_k=20, max_new_tokens=256):
- Mean first-token KL divergence (abliterated || source): 0.0323 — below Heretic's
<0.05"minimal damage" threshold. - JSON / tool-call validity: 100% (same as source).
- No AI-disclaimer tics, no trailing repetition.
- Distinct-2 ratios at or above the source model across QA, code, and reasoning tests.
Full data: eval/comparison.md, eval/comparison.json, eval/quantization.md.
Refusal Audit
Independent refusal/rejection audit across GGUF quantisations:
| Quantisation | High-risk refusal rate | Task accuracy |
|---|---|---|
q4_K_M | ~4% | ~52% |
q8_0 | ~2% | ~51% |
q4_K_S | ~0% | ~49% |
Full reports: refusal_audit_q4ks/ and refusal_audit_q80/ (markdown report plus raw
JSON results).
Repository Contents
model_files/— full-precision safetensors weights + tokenizer/config (model-0000N-of-00004.safetensors,model.safetensors.index.json,tokenizer.json,config.json, etc.).gguf/— quantised GGUF variants for Ollama / llama.cpp (q4_K_S,q4_K_M,q8_0).eval/— capability-retention and quantisation comparison reports.refusal_audit_q4ks/,refusal_audit_q80/— refusal-audit reports and raw results.LICENSE— Apache-2.0 licence (original + derivative notice).
Recommended quantization by hardware
All three quantizations fit comfortably in 16 GB+ of memory (weights + context). The
right pick is a quality / speed / thermals trade-off. Speeds in eval/quantization.md
were measured on the original (Windows + Ollama) test machine, so treat them as relative
guidance rather than exact numbers for your card.
| Hardware | Memory | Pick | Why |
|---|---|---|---|
| Apple M5 MacBook Air, 24 GB | 24 GB unified, fanless | q4_K_M | 5 GB footprint, fastest, lowest sustained heat — safest for a fanless Air under long generation runs. |
| NVIDIA RTX 4090 | 24 GB GDDR6X | q8_0 (quality) / q4_K_M (speed) | Fits with headroom; high bandwidth + active cooling keeps q8_0 fast for maximum quality. |
| NVIDIA RTX 5060 Ti, 16 GB | 16 GB GDDR7 | q4_K_M (balanced) / q8_0 (max quality) | q8_0 (8.7 GB) still fits 16 GB with room for context; on a mid-range card q4_K_M is the best speed/quality balance. |
Refusal behaviour is near-zero for all three (keyword TPR: q4_K_S 0%, q8_0 2%,
q4_K_M 4%), so the choice does not affect safety/uncensored behaviour — it is purely
about quality vs. speed vs. thermals.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "kalpakjian/qwen3-8b-heretic"
tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="model_files")
model = AutoModelForCausalLM.from_pretrained(
model_id, subfolder="model_files", device_map="auto"
)
Or with Ollama / llama.cpp, point a Modelfile at one of the gguf/*.gguf files.
License
Apache-2.0 — the same licence as the base Qwen/Qwen3-8B model. This is an
unofficial, user-generated derivative; see LICENSE for the derivative notice.
References
- Base model: Qwen/Qwen3-8B
- Capability comparison:
eval/comparison.md - Refusal audit:
refusal_audit_q4ks/test_report.md