Gemma-4-12B-QAT-StyleTune-Voice
Google's QAT base + Gryphe's 12B StyleTune voice. Best of both. One 2.0 GB download. Zero extra models.
This voice is cast from Gryphe/Gemma-4-12B-StyleTune — 56% fewer clichés (1.050 → 0.463 per 100 words), 16.8% shared trigram vocabulary vs base instruct — and matched to Google's QAT checkpoint, google/gemma-4-12B-it-qat-q4_0-unquantized. If your GGUF is built from the QAT weights, this is the variant that fits it exactly.
Why QAT + StyleTune is the good combo
Two independent optimizations, now combined:
Google's QAT trains the model to survive quantization. Per Google, QAT gives 54% less perplexity drop at Q4_0 versus standard post-training quantization (Google Developers Blog) — 5,000 steps of fake-quantized forward passes with KL divergence to the BF16 teacher. Your Q4_0 GGUF keeps noticeably more of the original model's fidelity — and at 12B, every bit of fidelity counts double.
StyleTune then changes how it writes: per Gryphe's benchmarks — 200 diverse roleplay prompts, greedy 0.0 — 56% fewer clichés with only 16.8% shared vocabulary vs the base instruct.
The old tradeoff was: QAT model or StyleTune model, pick one. Now you don't pick.
Two steps
# 1. Get the voice tool (one-time): https://huggingface.co/Wiself/voice
python3 voice.py path
# 2. Cast onto your QAT-derived GGUF
voice cast ./gemma-4-12b-it-qat-q4_0-Q4_0.gguf voice.safetensors --out ./voiced/gemma-4-12b-qat-styletune.gguf
Append --speak to force the voice through output.weight (invented if the target lacks it and geometry survives; GGUF only).
Run it:
llama serve -m ./voiced/gemma-4-12b-qat-styletune.gguf --jinja
No QAT finetune was needed. No extra model was downloaded.
Why the QAT-matched variant exists
The QAT checkpoint's lm_head weights differ slightly from the standard instruct's (that's the point of QAT — the weights learned to live with quantization). This voice is a delta against the normal google/gemma-4-12B-it (voice − base where base is the standard instruct, not QAT). When you later do delta + QAT_head → Q8_0, you add StyleTune's style to the QAT head — correct math, matched foundation.
| Your GGUF is built from | Use |
|---|---|
google/gemma-4-12B-it (standard) | 12B voice |
google/gemma-4-12B-it-qat-q4_0-unquantized | this voice |
If it loops on abliterated targets
On some abliterated targets a direct cast can loop. The fix is the delta path (base is the normal instruct — correct math):
voice delta voice.safetensors --base google/gemma-4-12B-it
voice cast ./model.gguf delta-voice.safetensors --out ./voiced/model.gguf
This is the exact path that fixed looping on the 26B QAT abliterated model — same recipe, 12B scale. Full story in the Voice tool card.
What's inside
voice.safetensors— the style delta, F16, shape[262144, 3840], ~2.0 GBvoice.json— metadata: source, dtype, shape, base (google/gemma-4-12B-it)
Compatibility
| Target | Works? |
|---|---|
| QAT-derived Gemma 4 12B GGUFs (any quant) | ✅ primary target |
| Abliterated QAT 12B variants | ✅ via the delta path above |
| Standard (non-QAT) 12B GGUFs | ✅ works, but the 12B voice is the exact match |
| Gemma 4 other sizes / non-Gemma | ❌ shape mismatch — use the matching voice |
Notes
- Casting quantizes only the head to Q8_0 (near-lossless); every other tensor is byte-copied from your model — your QAT weights stay QAT weights.
- Sampler tips from Gryphe: temp 1.0, MinP 0.10, DRY sampler on. Gemma 4's native chat template applies automatically.
- Verify:
voice info voice.safetensors→ delta marker,[262144, 3840] · F16. - Family: 12B voice · 26B A4B V2 voice · 26B A4B QAT voice · 31B voice · 31B QAT voice · 12B QAT voice (this repo)
References & Credits
- QAT: Google Developers Blog — Gemma 3 Quantization-Aware Training — 54% less perplexity drop at Q4_0; checkpoint
google/gemma-4-12B-it-qat-q4_0-unquantized, Gemma terms. - StyleTune: Gryphe/Gemma-4-12B-StyleTune — 56% cliché reduction, 16.8% shared trigram vocabulary; Anthracite, Latitude.
- Tool: Voice — lift a voice, cast it onto any compatible GGUF.