SuperGemma-4-12b-abliterated
SuperGemma-4-12b-abliterated is a fused 12B checkpoint derived from google/gemma-4-12B-it.
The release combines two post-training stages:
- Abliteration pass - a weight-space refusal-direction pass designed to suppress unnecessary refusal behavior and improve direct task completion.
- Supertune post-training - targeted post-training for instruction following, coding, Korean technical answers, JSON/tool formatting, and regression resistance.
The result is a single checkpoint with no runtime adapter requirement.
Benchmark Growth
The comparison target is the original Gemma4 12B instruction checkpoint.
| Benchmark | Gemma4 12B original | SuperGemma-4-12b-abliterated | Delta |
|---|---|---|---|
| Overall public top-5 500 | 23.8 | 44.6 | +20.8 |
| GPQA Diamond | 10.0 | 19.0 | +9.0 |
| MMLU-Pro | 17.0 | 18.0 | +1.0 |
| IFEval | 61.0 | 59.0 | -2.0 |
| HumanEval+ | 18.0 | 46.0 | +28.0 |
| MBPP+ | 13.0 | 81.0 | +68.0 |
Additional internal validation:
| Check | Result |
|---|---|
| Public benchmark prompts completed | 500 / 500 |
| Blank response ratio | 0.0 |
| Hidden-thought leak ratio | 0.0 |
| Release-surface response audit findings | 0 |
| Release bugcheck | 6 / 6 |
| Quickbench full20 overall | 95.4 |
| Mega 103 overall | 88.7 |
| Original reference Mega 103 overall | 84.1 |
Quantized Variants
Quantized builds are published as separate Hub repos and are linked back to this model through Hub metadata.
| Variant | Repository | Notes |
|---|---|---|
| Original BF16 | Jiunsong/SuperGemma-4-12b-abliterated | This repository |
| NVF4 / NVFP4 4-bit | Jiunsong/SuperGemma-4-12b-abliterated-nvf4 | MLX NVFP4 4-bit quantization |
| MLX 4-bit | Jiunsong/SuperGemma-4-12b-abliterated-mlx-4bit | MLX affine 4-bit quantization |
| GGUF 4-bit | Jiunsong/SuperGemma-4-12b-abliterated-gguf-4bit | llama.cpp GGUF Q4_K_M |
Usage
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "Jiunsong/SuperGemma-4-12b-abliterated"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
trust_remote_code=True,
)
messages = [{"role": "user", "content": "Write a concise release checklist."}]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_dict=True,
return_tensors="pt",
add_generation_prompt=True,
enable_thinking=False,
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Notes
- This release is optimized for direct task completion and reduced unnecessary refusals.
- Benchmarks are measured against the original Gemma4 12B instruction checkpoint using the recorded 500-prompt public top-5 suite.
- Use the quantized builds when runtime size is more important than exact BF16 fidelity.