Gemma-4-E2B-Heretic — GGUF Quants
Quantized GGUF versions of DuoNeural/Gemma-4-E2B-Heretic, DuoNeural's abliterated Gemma 4 2B Instruct variant.
The Heretic series applies norm-preserving orthogonal projection (abliteration) to surgically remove refusal behavior while preserving the full instruction-following and reasoning capabilities of the base model.
Available Files
| File | Quant | Size | Use Case |
|---|---|---|---|
Gemma-4-E2B-Heretic-Q8_0.gguf | Q8_0 | ~4.7GB | Maximum quality, ample VRAM |
Gemma-4-E2B-Heretic-Q6_K.gguf | Q6_K | ~3.6GB | Near-lossless, recommended for 6GB+ |
Gemma-4-E2B-Heretic-Q5_K_M.gguf | Q5_K_M | ~3.4GB | High quality, good balance |
Gemma-4-E2B-Heretic-Q4_K_M.gguf | Q4_K_M | ~3.2GB | Recommended default — quality/size sweet spot |
Gemma-4-E2B-Heretic-Q3_K_M.gguf | Q3_K_M | ~3.0GB | Low VRAM, acceptable quality |
Gemma-4-E2B-Heretic-IQ4_XS.gguf | IQ4_XS | ~3.1GB | Imatrix-optimized 4-bit |
Gemma-4-E2B-Heretic-IQ3_XXS.gguf | IQ3_XXS | ~2.5GB | Imatrix 3-bit, aggressive compression |
Gemma-4-E2B-Heretic-IQ2_M.gguf | IQ2_M | ~2.4GB | Imatrix 2-bit, minimum viable quality |
Gemma-4-E2B-Heretic-IQ1_S.gguf | IQ1_S | ~2.2GB | Extreme compression, experimental |
E2B-Heretic-fp16.gguf | FP16 | ~8.7GB | Full precision base for re-quantization |
imatrix.dat | — | — | Importance matrix for custom IQ quants |
Usage
llama.cpp
./llama-cli -m Gemma-4-E2B-Heretic-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--ctx-size 8192 -n 512
Ollama
ollama run hf.co/DuoNeural/Gemma-4-E2B-Heretic-GGUF:Q4_K_M
Jan / LM Studio / Open WebUI
Download and load the .gguf file directly. Recommended: Q4_K_M for most hardware.
About Heretic Abliteration
The Heretic series uses DuoNeural's 2-pass abliteration pipeline:
- Residual collection — identify refusal direction in activation space using 4-bit GPU inference
- Weight modification — project out the refusal direction from MLP/attention weight matrices in bf16 on CPU
- Validation — confirm compliance behavior removed, helpfulness preserved
The base model is google/gemma-4-e2b-it — Google's Gemma 4 2B instruction-tuned model with 128K context, multimodal inputs, and strong instruction-following capability.
Notes
- IQ formats use the included
imatrix.datfor importance-weighted quantization - For custom IQ quants:
llama-quantize --imatrix imatrix.dat model.gguf output.gguf IQ4_XS - AWQ format not available: AutoAWQ does not currently support the Gemma 4 architecture
Quantized by DuoNeural using llama.cpp on RTX 5090.
DuoNeural
DuoNeural is an open AI research lab — human + AI in collaboration.
| Platform | Link |
|---|---|
| HuggingFace | huggingface.co/DuoNeural |
| Website | duoneural.com |
| GitHub | github.com/DuoNeural |
| X / Twitter | @DuoNeural |
| duoneural@proton.me | |
| Newsletter | duoneural.beehiiv.com |
| Support | buymeacoffee.com/duoneural |
DuoNeural Research Publications
Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.