Qwen2.5-72B-Instruct-heretic
An abliterated (decensored) version of
Qwen/Qwen2.5-72B-Instruct,
produced with Heretic via the one-command
heretic_fork runner.
Abliteration identifies and removes the directions in the model's residual stream most associated with refusal behaviour, using a difference-of-means direction between a set of "harmless" and "refused" prompts, then merges the resulting low-rank correction back into the weights. The result is a standard, fully-merged model directory — no adapter, no special loader.
Results
Measured on a held-out evaluation set (66 refusal prompts, 66 harmless prompts) that was not used to compute the ablation direction:
| Metric | Original model | Abliterated model |
|---|---|---|
| Refusals (keyword) | 56 / 66 | 0 / 66 |
| KL divergence | 0 (by definition) | 0.0035 |
Every refusal on the held-out set was removed, while the KL divergence from the original model is 0.0035 — far below the ~0.5 threshold that typically indicates capability damage. In other words, refusals are gone with no measurable degradation of the model's general behaviour.
- Trials: 120 (40 random startup), TPE multi-objective optimization
- Ablated components:
attn.o_proj,mlp.down_projacross all 80 layers - Precision: bf16, merged weights
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "krthkyadtya/Qwen2.5-72B-Instruct-heretic"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", dtype="auto")
messages = [{"role": "user", "content": "Your prompt here"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
License
Inherits the Qwen license of the base model. The Heretic tooling is AGPL-3.0-or-later.
Note
This model has had its safety refusal behaviour removed and will comply with requests the original model would decline. Use responsibly and in accordance with applicable law and the base model's license.