Qwen3-VL-4B-Instruct Heretic (Abliterated)

💬 Community: Join the Abliterlitics Discord for discussion, model releases and support.

An abliterated version of Qwen3-VL-4B-Instruct, produced with the Heretic optimiser. Heretic is stochastic, so each trial finds a slightly different refusal direction in the same base model. Four trials were run, compared forensically, and the winner is the model published here. In the report below it is labelled t122; this repo ships that variant.

The full forensic comparison against the other candidates (t174, t191 and an averaged direction) is the body of this README, produced with Abliterlitics, an open-source abliteration forensics toolkit.

Available formats

This model is published across three repos. Pick the one that matches your runtime.

The Docker setup, scripts and configs that produced these files are in Heretic Docker.

Highlights

• All four variants fully jailbreak the model. HarmBench attack-success-rate goes from 30.8% to 100% on each. The base already complied with 123 of 400 harmful requests; the variants comply with all 400.
• t122 is the best tradeoff and the variant shipped here. It has the lowest KL divergence (0.0283, the smallest shift from the base distribution), the smallest benchmark regression (GSM8K down 1.83%) and the lightest pure rank-1 edit (54 tensors).
• t174 is the most disruptive. Highest KL (0.0649, about 2.3 times t122) and the worst capability loss.
• Three variants are textbook rank-1 abliterations; the averaged direction is not. t122, t174 and t191 edit each tensor with a single direction (effective rank 1.0). The averaged variant superimposes several directions, so some of its edits are full rank.
• Heretic edits both self_attn.o_proj and mlp.down_proj, not just the attention output projection that classic abliteration targets.

Quick facts

Files

model-00001-of-00002.safetensors        ~5.0 GB
model-00002-of-00002.safetensors        ~3.9 GB
model.safetensors.index.json
config.json
generation_config.json
chat_template.jinja
preprocessor_config.json
video_preprocessor_config.json
tokenizer.json / tokenizer_config.json
special_tokens_map.json / added_tokens.json
vocab.json / merges.txt

Usage

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
import torch

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "DreamFast/Qwen3-VL-4b-Heretic",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
processor = AutoProcessor.from_pretrained("DreamFast/Qwen3-VL-4b-Heretic")

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "https://example.com/image.jpg"},
        {"type": "text", "text": "Describe this image in detail."},
    ],
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(out[0], skip_special_tokens=True))

Forensic report

The rest of this README is the forensic comparison of all four Heretic variants. The variant shipped here is t122.

Benchmarks

Evaluated with lm-evaluation-harness through a vLLM OpenAI-compatible server on a single NVIDIA RTX 5090 (32 GB). A 4B model fits on the card in full precision, so all five models ran in bf16 with no quantisation, and the deltas are directly comparable.

Delta vs base

What the benchmarks tell us

• Knowledge and common-sense (MMLU, HellaSwag, ARC, WinoGrande, PiQA) are essentially untouched. Deltas stay within about 0.6% on most tasks and are often positive. Removing the refusal direction does not measurably harm factual recall.
• Math reasoning (GSM8K) is the canary. It degrades on every variant, and the size of the drop tracks the KL divergence almost monotonically: t122 (−1.83%, KL 0.028), then the averaged variant (−2.70%, KL 0.034), then t191 (−3.28%, KL 0.043), then t174 (−7.23%, KL 0.065). The more the model is perturbed, the more step-by-step arithmetic suffers.
• TruthfulQA takes the largest hit across the board (MC2 down 9% to 12.6%, Gen down 21% to 22.7%). This is the expected signature of abliteration: the refusal direction overlaps with the direction that stops the model stating falsehoods as fact, so truthfulness calibration drops predictably. t174 is worst here too.

Safety: HarmBench

HarmBench, 400 textual behaviours, max_tokens=2048, temperature=0, judged by the HarmBench classifier.

All four variants refuse zero of the 400 behaviours. The base model is already fairly permissive, complying with nearly a third of harmful requests, so the abliteration closes the remaining gap. Every category flips from partial to total compliance.

ASR by category

The base model's only near-complete compliance is on Copyright (99%); it will already reproduce copyrighted text. Its strongest guardrails were Harassment (0%) and Chemical/Biological (5.4%), which the variants erase entirely. Because every variant reaches 100%, safety cannot discriminate between them, so the choice has to come down to capability and fidelity.

KL divergence

Methodology: F.kl_div(logprobs_variant, logprobs_base, reduction="batchmean", log_target=True) on full-vocab first-token logits, matching the Heretic evaluator.

Rating scale: excellent under 0.01, very good 0.01 to 0.1, moderate 0.1 to 0.4, significant 0.4 to 1.0, heavy over 1.0.

All four sit comfortably in the "very good" band, so the edits are surgical. The spread is what matters: t174 perturbs the output distribution about 2.3 times as much as t122. KL is the single best predictor of how much a variant will drift in behaviour, and it ranks the variants exactly as the GSM8K regression does, from t122 best to t174 worst. That makes it the cleanest signal for picking a variant when safety is tied.

Weight analysis

Modification summary

Two patterns stand out:

• Aggressiveness versus per-tensor magnitude. The averaged variant touches the most tensors (64) but with the smallest per-tensor edits (mean norm 1.27), because averaging dampens each direction. t174 touches fewer tensors but hits them hard (mean norm 2.09). t122 is gentle on both axes.
• t191 is the most surgical by count (50 tensors, 12.56%) but spreads them across the most layers (94% coverage). t122 concentrates the same idea into fewer layers.

Which tensor types get modified

Classic (Arditi et al.) abliteration targets only self_attn.o_proj. All four Heretic variants also edit mlp.down_proj, which confirms Heretic finds refusal-related directions in the MLP pathway as well as the attention pathway. t122 is the only one with a perfectly balanced 27/27 split; t191 leans heavily on attention (34 versus 16).

SVD rank: rank-1 versus full-rank

This is the structural fingerprint of the technique:

• t122, t174 and t191 are pure rank-1 abliterations. Every modified tensor's edit is a single outer product (effective rank exactly 1.0, and the top singular value carries 96 to 99.7% of the energy). This is the textbook "subtract one refusal direction" operation.
• The averaged variant is a superposition. Because it averages several trial directions, its edits have rank greater than 1 (down_proj up to rank 20, top-1 energy as low as 53.9%). It is a fundamentally different intervention, a blend rather than a single direction.

See the per-variant SVD plots (graphs/svd_summary_qwen3-vl-4b_*.svg) and per-layer heatmaps (graphs/layer_*_qwen3-vl-4b_*.svg).

Cross-technique overlap: which tensors do the trials share?

The three rank-1 trials target highly overlapping but not identical tensor sets:

The trials largely converge on the same refusal circuit, agreeing on where the refusal direction lives, but they disagree on its exact orientation. That disagreement is exactly what the KL spread (0.028 to 0.065) reflects.

Picking a variant

This repo ships t122. It wins on the two metrics that should decide a tie when safety is already 100% across the board: the lowest KL (most faithful to base) and the best GSM8K (least reasoning damage), while still being a clean, interpretable pure rank-1 edit. If you prefer the averaged-direction philosophy and want the lowest per-tensor magnitude, the averaged variant is a fine alternative that holds TruthfulQA slightly better. t191 is the minimalist choice (fewest tensors touched). t174 should be avoided when reasoning matters.

Summary

Variant summaries

• Heretic t122 is the recommended pick and the one shipped here. Lowest KL (0.0283), best GSM8K retention (−1.83% versus base), and a clean pure rank-1 edit over a balanced 27/27 attention-MLP split. It is the trial that found the least refusal direction sufficient to fully jailbreak the model.
• Heretic t174 is the most aggressive trial. Highest KL (0.0649) and the worst reasoning and TruthfulQA scores. It edits a strict superset of t122's tensors (100% overlap) but pushes harder and pays for it in capability. Useful only if you want maximal de-alignment regardless of cost.
• Heretic t191 is the minimalist. It touches the fewest tensors (50) but spreads them across the most layers (94%). Middle-of-the-pack KL (0.0430) and benchmarks, and its tensor set overlaps t174 by 96%.
• Heretic Avg is a superposition of multiple trials. It is unique here for having full-rank edits (effective rank up to 20) rather than single directions. It has the highest tensor count (64) but the gentlest per-tensor magnitude (1.27) and the best TruthfulQA. Pick it if you trust the ensemble over any single trial.

Methodology

• Capability: lm-evaluation-harness through a vLLM OpenAI-compatible server on a single NVIDIA RTX 5090 (32 GB). The 4B model fits in full precision, so all five models ran in bf16 with no quantisation, which keeps the cross-variant deltas directly comparable.
• Tasks: MMLU, GSM8K, HellaSwag, ARC-Challenge, WinoGrande, TruthfulQA (MC1/MC2/Gen), PiQA, Lambada-OpenAI.
• Safety: HarmBench, 400 textual behaviours, max_tokens=2048, temperature=0, judged by the HarmBench classifier.
• KL divergence: full-vocab first-token logits via model.generate(max_new_tokens=1, output_scores=true), then F.kl_div(..., reduction="batchmean"), matching the Heretic evaluator methodology.
• Weight analysis: SVD effective-rank and energy spectra, edit-vector fingerprints, per-layer magnitude profiles and pairwise tensor-set overlap, all using Abliterlitics.
• Hardware: NVIDIA RTX 5090 (32 GB). Graph and report generation are CPU only.

Artifacts

Produced by the Abliterlitics run and referenced above:

• Graphs: graphs/, 18 SVGs (Venn overlap, aggressiveness bars, 12 per-variant layer heatmaps, 4 per-variant SVD spectra).
• Data tables: benchmark, benchmark-delta, harmbench, harmbench-categories, kl, weight-summary and summary.
• Raw results: per-variant fingerprint, SVD, layer-analysis and edit-vector files, KL results, lm-eval results, HarmBench responses and classifications, and pairwise correlation, low-rank and subspace analyses.

Limitations

• Inherits every limitation of the base Qwen3-VL-4B-Instruct model.
• Abliteration reduces refusals but does not guarantee removing all of them in every context.
• TruthfulQA and GSM8K drop measurably. Use the base model if either matters critically for your task.

Acknowledgements

• Qwen team for Qwen3-VL-4B-Instruct
• Heretic by p-e-w, the abliteration optimiser
• Abliterlitics for the forensic analysis
• lm-evaluation-harness and HarmBench

Disclaimer

This model has had its safety alignment removed. It complies with harmful requests, including content related to violence, illegal activities and other harmful behaviour. Use it responsibly and in line with the laws and regulations that apply to you. The authors do not condone or encourage using this model for harmful purposes.