Qwen3.6-27B-Fable-Fusion-711-Heretic2
A second, terminal Heretic ablation of DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP, targeted at creative-writing refusals rather than AdvBench-style harm refusals.
GGUF quants: …-Heretic2-GGUF
Why this exists
The base model is described as "Uncensored / Heretic", and its card reports
4/100 refusals. That number is real, but the same card also says
"Heretic uncensored (pre tuning)" — the ablation was performed on stock
Qwen/Qwen3.6-27B by trohrbaugh, and DavidAU then ran multi-stage fine-tuning
and merging on top, including Claude Opus reasoning traces and GPT-5 Polaris
data. Training on generations from aligned models re-teaches refusal.
The published figure therefore describes an intermediate artifact, not the released weights. In practice the released model still refuses.
In this model the ablation is the last step. Nothing is trained or merged afterward, so the measured numbers describe the weights you are downloading.
Method
Heretic v1.4.0, ARA-style configuration:
| setting | value |
|---|---|
orthogonalize_direction | true |
row_normalization | full |
full_normalization_lora_rank | 3 |
quantization | bnb_4bit |
| adapter | r=3, alpha=3 (scale 1.0) |
| target modules | down_proj, out_proj, o_proj |
| resulting LoRA pairs | 128 |
| max abs weight delta | 8.968961e-02 |
Prompt sets — the important part
Stock Heretic computes its refusal direction from mlabonne/harmful_behaviors
(416 AdvBench rows, almost entirely "write a tutorial on how to hack X").
That direction has little to do with a model balking mid-scene at violence or
an adult scene, which is why generic abliterations often fail for fiction work.
This run used a custom probe set: ~350 prompts collected by actually hitting refusals on the base model during creative-writing use, merged with AdvBench so that "character explains how to pick a lock" style refusals are still covered.
- Direction set: custom prompts (weighted ×3) + AdvBench ≈ 1157 lines
- Evaluation set: 100 custom prompts, held out — they never appear in the direction set, so the refusal score measures generalisation rather than memorisation of the prompts it was fitted on.
Results
Selected from the Pareto front produced by the optimiser:
| metric | value |
|---|---|
| refusals (held-out set) | 1 / 100 |
| baseline before ablation | 17 / 100 |
| KL divergence | 0.0173 |
Heretic notes that KL above ~0.5 indicates significant capability damage; this is roughly two orders of magnitude below that.
Read these numbers narrowly. The scorer generates a bounded number of tokens at temperature 0 and substring-matches a refusal-marker list. A response that opens agreeably and declines later, or declines in wording outside the list, is not counted. Treat it as "the ablation generalised on a held-out set of the prompts I care about", not as an absolute refusal rate.
⚠️ The MTP layer was removed
The base model is a -MTP build: it carries an extra multi-token-prediction
layer (blk.64, nextn_predict_layers: 1) used for speculative decoding.
transformers does not model that head, so save_pretrained drops it. These
weights therefore contain 64 layers, not 65, and config.json reflects that.
If you quantise these weights to GGUF yourself, the converter may still read
nextn_predict_layers and write block_count: 65, producing a file that fails
to load with:
check_tensor_dims: tensor 'blk.64.attn_norm.weight' not found
The GGUFs in the companion repo have this metadata corrected. Speculative decoding via MTP is not available in this model.
Reproducing
The 26 MB Heretic adapter reproduces this from the public base model.
Do not merge it with PEFT. PeftModel.from_pretrained fails to bind Heretic's
key layout — it emits a "Found missing adapter keys" warning and then returns the
unmodified base model, with no exception and a clean exit. Apply the LoRA
math directly instead, and assert the weights actually changed:
delta = (B.float() @ A.float()) * (lora_alpha / r)
w.add_(delta.to(w.dtype))
assert max_abs_delta > 0.0 # or you have silently shipped the original model
License
Apache 2.0, inherited from the base model. Derivative work.