This is a decensored version of google/gemma-4-E2B-it, made using Heretic v1.4.0

Abliteration parameters

Performance

Reproducibility

| Setting | Value | | Base model | google/gemma-4-E2B-it | | Trials | 200 (n_startup_trials 60) | | Seed | 3407 | | dtype / quantization | auto (bfloat16) / none | | Row normalization | full (LoRA rank 3) | | Orthogonalize direction | yes | | Batch size | 16 (auto-selected, --max-batch-size 16) | | Selected trial | Pareto front trial with lowest refusals (21/100, KL 0.3269) |

[!NOTE] The randomized SVD (torch.svd_lowrank) inside Heretic's full row-normalization path was executed on the CPU rather than the MPS backend. The PyTorch MPS build of svd_lowrank takes minutes per call instead of milliseconds, which would have made each optimization trial take hours. The RNG is reseeded (torch.manual_seed(3407)) immediately before every call, so results remain deterministic and reproducible.

[!WARNING] These weight files have been modified.

This repository contains google/gemma-4-E2B-it after directional ablation ("abliteration"), performed automatically with Heretic v1.4.0. A refusal direction was identified in the model's residual stream and projected out of every attention out-projection (o_proj) and MLP down-projection (down_proj) across all transformer layers, with norm-preserving (full) row normalization, then merged into the base weights.

The model's safety alignment has been deliberately reduced. Refusals on sensitive prompts are substantially lower than the original checkpoint, and outputs are not safety-filtered. Use accordingly.

What changed

🏃 How to run

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model_id = "atakankutalp/gemma-4-E2B-it-heretic"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Who are you?"}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=128)
print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Dataset

• Good prompts (KL reference): mlabonne/harmless_alpaca (train[:400], evaluated on test[:100])
• Bad prompts (refusal evaluation): mlabonne/harmful_behaviors (train[:400], evaluated on test[:100])

GGUF (llama.cpp / llama.cpp-compatible runtimes)

Quantized GGUF builds of this exact ablated checkpoint, converted with llama.cpp's convert_hf_to_gguf.py and quantized with llama-quantize. Useful for Windows / CPU inference (Ollama, LM Studio, llama.cpp, koboldcpp, ...).

llama-cli -hf atakankutalp/gemma-4-E2B-it-heretic --conversation \
    --temp 0.1 --top-k 50 --repeat-penalty 1.1

[!NOTE] The GGUF files are text-only: the vision tower of Gemma4ForConditionalGeneration is not included. Use the safetensors files in this repository for multimodal use.