Gemma-4-31B-QAT-StyleTune-Voice

Google's QAT base + Gryphe's strongest StyleTune voice. Best of both. One 2.8 GB download. Zero extra models.

This voice is cast from Gryphe/Gemma-4-31B-StyleTune — the original StyleTune, with the family's largest cliché reduction (60%, 1.23 → 0.52 per 100 words, 21.7% shared trigram vocabulary) — and matched to Google's QAT checkpoint, google/gemma-4-31B-it-qat-q4_0-unquantized. If your GGUF is built from the QAT weights, this is the variant that fits it exactly.

Why QAT + StyleTune is the good combo

Two independent optimizations, now combined:

Google's QAT trains the model to survive quantization. Per Google, QAT gives 54% less perplexity drop at Q4_0 versus standard post-training quantization (Google Developers Blog) — 5,000 steps of fake-quantized forward passes with KL divergence to the BF16 teacher.

StyleTune then changes how it writes: per Gryphe's benchmarks — 200 roleplay prompts, greedy 0.0 — 60% fewer clichés with only 21.7% shared vocabulary vs the base instruct. The strongest de-slop effect in the family.

The old tradeoff was: QAT model or StyleTune model, pick one. Now you don't pick.

Two steps

# 1. Get the voice tool (one-time): https://huggingface.co/Wiself/voice
python3 voice.py path

# 2. Cast onto your QAT-derived GGUF
voice cast ./gemma-4-31b-it-qat-q4_0-Q4_0.gguf voice.safetensors --out ./voiced/gemma-4-31b-qat-styletune.gguf

Append --speak to force the voice through output.weight (invented if the target lacks it and geometry survives; GGUF only).

Run it:

llama serve -m ./voiced/gemma-4-31b-qat-styletune.gguf --jinja

No QAT finetune was needed. No extra model was downloaded.

Why the QAT-matched variant exists

The QAT checkpoint's lm_head weights differ slightly from the standard instruct's (that's the point of QAT — the weights learned to live with quantization). This voice is a delta against the normal google/gemma-4-31B-it (voice − base where base is the standard instruct, not QAT). When you later do delta + QAT_head → Q8_0, you add StyleTune's style to the QAT head — correct math, matched foundation.

The abliterated case (why we recommend delta by default here)

If your target is an abliterated/uncensored QAT variant and a direct cast loops, use the delta path (base is the normal instruct — correct math):

voice delta voice.safetensors --base google/gemma-4-31B-it
voice cast ./model.gguf delta-voice.safetensors --out ./voiced/model.gguf

Why it works: a delta carries only the difference from the base head, so the shift to MoE routing is minimal. This is the exact path that fixed looping on the 26B QAT abliterated model — same recipe, 31B scale.

What's inside

• voice.safetensors — the style delta, F16, shape [262144, 5376], ~2.8 GB
• voice.json — metadata: source, dtype, shape, base (google/gemma-4-31B-it)

Compatibility

Notes

• Casting quantizes only the head to Q8_0 (near-lossless); every other tensor is byte-copied from your model — your QAT weights stay QAT weights.
• Sampler tips from Gryphe: temp 1.0, MinP 0.10, DRY sampler on. Gemma 4's native chat template applies automatically.
• Verify: voice info voice.safetensors → delta marker, [262144, 5376] · F16.
• Family: 12B voice · 26B A4B V2 voice · 26B A4B QAT voice · 31B voice · 31B QAT voice (this repo)

References & Credits

• QAT: Google Developers Blog — Gemma 3 Quantization-Aware Training — 54% less perplexity drop at Q4_0; checkpoint google/gemma-4-31B-it-qat-q4_0-unquantized, Gemma terms.
• StyleTune: Gryphe/Gemma-4-31B-StyleTune — 60% cliché reduction, 21.7% shared trigram vocabulary; Anthracite, Latitude.
• Tool: Voice — lift a voice, cast it onto any compatible GGUF.