gemma-4-26B-A4B-it-GGUF Tater NoThink
This is a Tater NoThink test build of unsloth/gemma-4-26B-A4B-it-GGUF, bundled with compatible Multi-Token Prediction (MTP) and DFlash draft models for llama.cpp speculative decoding.
The main model weights and projector are from the upstream Unsloth GGUF release. The only intended change to the main text model is its embedded GGUF tokenizer.chat_template metadata:
gemma-4-26B-A4B-it-UD-Q4_K_M.gguf
The template is patched to force:
{%- set enable_thinking = false -%}
This makes the model prefer the no-thinking chat-template path even when a client accidentally exposes or passes a thinking flag.
Included Files
gemma-4-26B-A4B-it-UD-Q4_K_M.gguf: Tater NoThink patched GGUF.mmproj-F16.gguf: unchanged upstream vision projector.gemma-4-26B-A4B-it-MTP-Q8_0.gguf: Q8_0 MTP draft model converted from Google's official Gemma 4 assistant checkpoint. This is a sidecar draft model, not a standalone chat model.gemma-4-26B-A4B-it-DFlash-Q8_0.gguf: Q8_0 DFlash draft model from the exact Gemma 4 26B-A4B-it DFlash checkpoint. This is also a sidecar draft model, not a standalone chat model.
Speculative Decoding In Tater
Download the main model and one compatible draft model, then configure the llama.cpp provider in Tater.
MTP
- Select
gemma-4-26B-A4B-it-UD-Q4_K_M.ggufas the main model. - Enable Speculative Decoding.
- Set Method to Multi-Token Prediction (MTP).
- Select
gemma-4-26B-A4B-it-MTP-Q8_0.ggufunder Draft Model (GGUF). - Leave Maximum Draft Tokens at the recommended value of
3, then use Save & Load.
The draft must be paired with this Gemma 4 26B target family. It is not interchangeable with DFlash, DSpark, or drafts made for another target model.
DFlash
- Select
gemma-4-26B-A4B-it-UD-Q4_K_M.ggufas the main model. - Enable Speculative Decoding.
- Set Method to DFlash.
- Select
gemma-4-26B-A4B-it-DFlash-Q8_0.ggufunder Draft Model (GGUF). - Set Maximum Draft Tokens to
3, then use Save & Load.
The DFlash checkpoint was trained with a block size of 16, but its published mixed-workload benchmark found 3 to be the best general starting value. Structured tasks such as code or JSON may benefit from 4 or 5.
llama.cpp command-line example
Use a recent llama.cpp build with Gemma 4 MTP support:
llama-server \
--model gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
--model-draft gemma-4-26B-A4B-it-MTP-Q8_0.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--jinja \
--flash-attn on
For DFlash, replace the draft model and speculative type:
llama-server \
--model gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
--model-draft gemma-4-26B-A4B-it-DFlash-Q8_0.gguf \
--spec-type draft-dflash \
--spec-draft-n-max 3 \
--jinja \
--flash-attn on
Actual speedup depends on prompt, hardware, context length, and draft acceptance rate.
Source And Attribution
Upstream repository:
- https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF
- https://huggingface.co/google/gemma-4-26B-A4B-it-assistant
- https://huggingface.co/z-lab/gemma-4-26B-A4B-it-DFlash
- https://huggingface.co/williamliao/gemma-4-26B-A4B-it-DFlash-GGUF
License:
- Apache 2.0
This repo is meant as a practical Tater compatibility build, not a new model or fine tune.
Draft model provenance
The MTP sidecar was converted from google/gemma-4-26B-A4B-it-assistant revision 6e5aaaf4c42b98394530b8fda2e95cadd65c151c with llama.cpp revision 8e7f22b67ef4, using Q8_0 output. During conversion, the source tokenizer configuration's empty extra_special_tokens list was treated as an empty mapping to satisfy the converter; model weights were not changed.
The DFlash sidecar is redistributed unchanged from williamliao/gemma-4-26B-A4B-it-DFlash-GGUF revision d1800ac59f255542ae096018fa696f03918066a6. That conversion is derived from the Apache-2.0-licensed z-lab/gemma-4-26B-A4B-it-DFlash checkpoint and was tested by its publisher with llama.cpp and the same Gemma 4 26B-A4B-it target family.
| File | Size | SHA-256 |
|---|---|---|
gemma-4-26B-A4B-it-MTP-Q8_0.gguf | 461,766,592 bytes | 9764ab8276181017bf565c2be9e7c61cc59b6f74f877c3d1514ca36e01d040f1 |
gemma-4-26B-A4B-it-DFlash-Q8_0.gguf | 472,432,704 bytes | 48ecabebc399e5424b89913197204f80cda23e42228c67b6725f1ab1cc8f5da3 |
DSpark Status
A Gemma 4 26B-A4B-it DSpark GGUF exists, but its current SpecForge conversion requires llama.cpp PR #26275 and the dflash.bonus_anchor runtime behavior. That support is not present in the llama.cpp revision bundled with the current Tater release, so the DSpark file is intentionally not mirrored here yet. It will be suitable to add after Tater moves to a compatible llama.cpp revision and passes a paired-model test.
Tater Usage
Use this repo as a llama.cpp GGUF model in Tater. For vision, select mmproj-F16.gguf as the projector.
Recommended first file:
gemma-4-26B-A4B-it-UD-Q4_K_M.gguf