Qwen3.5-4B-MTP-Heretic
Qwen/Qwen3.5-4B with its refusal behaviour ablated using
heretic v1.4.0, and with the multi-token-prediction
head left intact.
Why this model exists
I wanted an abliterated Qwen3.5-4B that was still a complete vision-language model and still had its MTP head working, and that documented clearly enough to be checked. Neither property is guaranteed by the toolchain. heretic's merge export drops the MTP head silently (see below), and the published Qwen3.5-4B abliterations I found gave no clear account of how they had been done — which refusal direction, what weight range, which trial, what the utility cost — so there was no way to tell a careful ablation from a reckless one.
So this is an attempt at a variant you can actually check: the exact heretic configuration, the trial that was selected, the measured refusal rate before and after, and a note on every step that is easy to get silently wrong.
⚠️ Risk of Sensitive or Controversial Outputs — This model's safety filtering has been significantly reduced. It will comply with requests that a stock Qwen3.5-4B refuses, including harmful ones. You are responsible for what you do with it.
⚠️ No Default Safety Guarantees — Unlike stock Qwen3.5-4B, this model has not undergone safety optimization. Do not expose it to untrusted input without a policy layer of your own.
What was done
heretic searches for a low-rank "refusal direction" in the residual stream and orthogonalises
the attn.o_proj and mlp.down_proj output projections against it, so the model can no
longer represent a refusal. It optimises a weight vector over 200 trials against
mlabonne/harmless_alpaca (good prompts, preserve utility) and mlabonne/harmful_behaviors
(bad prompts, remove refusal), maximising refusal reduction subject to a KL-divergence
budget against the original model.
The published checkpoint is trial 109 — the lowest-refusal point on the Pareto front:
| value | |
|---|---|
| refusals (heretic, in-run) | 19 / 100 |
| KL divergence vs base | 0.0459 |
direction_index | 20.24 |
attn.o_proj weights | 1.36 – 1.49, centred at layer 20.64 |
mlp.down_proj weights | 1.33 – 1.40, centred at layer 21.71 |
| trials / seed | 200, seed=0 |
The edit is a weight orthogonalisation, not retraining. Nothing was fine-tuned, and no data was trained on.
Results
Refusal rate over the first 100 prompts of mlabonne/harmful_behaviors (temperature 0,
max_tokens 100, system prompt "You are a helpful assistant."):
| model | refusals |
|---|---|
Qwen/Qwen3.5-4B (stock) | 99 / 100 |
| this model | 13 / 100 |
Measured with llama-server and a keyword-based refusal classifier, not a proper
capability benchmark — treat it as a relative signal only. A second variant of this model
that suppresses the thinking block reached 10 / 100 and is published separately.
The MTP head is preserved
This is the part that distinguishes it from other published Qwen3.5-4B abliterations.
Qwen3.5-4B ships a multi-token-prediction head (mtp_num_hidden_layers = 1, 15 tensors,
visible as block 32 in the GGUF). heretic's --export-strategy merge does not write those
tensors — it saves the 723 tensors it loaded and the MTP head is silently gone, with no
warning. Converted naively you get block_count = 32 and speculative decoding quietly stops
working.
| model | GGUF tensors | block_count |
|---|---|---|
Qwen/Qwen3.5-4B (stock) | 441 | 33 |
| this model | 441 | 33 |
Qwen3.5-4B-Deckard-HERETIC-UNCENSORED-Thinking | 426 | 32 |
The 15 missing tensors in that last row are the MTP head. This checkpoint has them re-injected and the index corrected, so the drafter is intact.
Format
Standard transformers format, 3 safetensors shards (9.3 GB total):
config.json generation_config.json chat_template.jinja
model-00001-of-00003.safetensors model-00002-of-00003.safetensors
model-00003-of-00003.safetensors model.safetensors.index.json
preprocessor_config.json processor_config.json video_preprocessor_config.json
tokenizer.json tokenizer_config.json LICENSE
738 tensors: 297 vision, 426 language, 15 MTP.
Architecture is unchanged (Qwen3_5ForConditionalGeneration), so it is still a
vision-language model and the processor configs are the stock ones. Only the
o_proj / down_proj weights of the language model differ from the base.
Mixed precision, deliberately. 723 tensors are float16 (heretic was run with
--dtypes float16, which is what was stable on ROCm for this model). The 15 re-injected MTP
tensors are still bfloat16, copied verbatim from the base checkpoint. config.json
declares dtype: float16, so from_pretrained casts them on load and this is transparent —
but if you load the shards by hand, expect the two dtypes.
GGUF
A Q8_0 GGUF plus the f16 vision projector are published in the companion repository
RBergBauer/Qwen3.5-4B-MTP-Heretic-GGUF
for LM Studio / llama.cpp. general.name is set to Qwen3.5-4B-MTP-Heretic.
# with the vision projector (required for image input)
llama-server -m Qwen3.5-4B-MTP-Heretic-Q8_0.gguf --mmproj mmproj-F16.gguf -ngl 99
The chat template is the stock Qwen3.5 one and is still parameterised on
enable_thinking, so the thinking block behaves as upstream.
Intended use
Research on refusal directions and model alignment; local experimentation with reduced safety alignment; red-teaming and evaluation harness development. Not intended for deployment where untrusted users can reach the model without a filtering layer.
Reproduction
heretic 1.4.0, llama.cpp b10689, seed=0, n_trials=200, fp16. The one step that is easy
to miss: the MTP tensors must be re-injected from the base model after every heretic save, or
they are silently absent from the export.
License
Apache 2.0, inherited from Qwen/Qwen3.5-4B. See LICENSE.