gemma-4-26B-A4B-it-qat-GGUF

google/gemma-4-26B-A4B-it-qat-q4_0-unquantized is a Mixture-of-Experts (MoE) instruction-tuned multimodal model from Google DeepMind, part of the Gemma 4 family, featuring 25.2 billion total parameters but only 3.8 billion active parameters during inference, optimized via Quantization-Aware Training (QAT) to preserve near-bfloat16 quality at significantly reduced memory requirements. Its sparse MoE architecture activates just 4B of its 128 total experts (plus 1 shared) per token across 30 layers with a 1024-token sliding window, making it run nearly as fast as a dedicated 4B model while delivering quality competitive with much larger dense models — scoring an impressive 82.6% on MMLU Pro, 82.3% on GPQA Diamond, 88.3% on AIME 2026, 77.1% on LiveCodeBench v6, 73.8% on MMMU Pro (vision), and 44.1% on the 256K long-context MRCR v2 task. Supporting text and image modalities (no audio) with a 256K token context window, a ~550M parameter vision encoder, and a 262K vocabulary across 140+ languages, the model enables image understanding, OCR, video frame analysis, native function calling, and configurable thinking/reasoning mode, with the Q4_0 unquantized variant providing half-precision weights extracted from the QAT pipeline, making it ideal for custom downstream compilation and research targeting high-throughput, cost-efficient server-side deployment.

[!note] Google DeepMind’s Gemma 4 Quantization-Aware Training (QAT) releases compress models by simulating lower precision during the training process itself. This drastically reduces VRAM requirements and accelerates local inference on consumer hardware and mobile devices while preserving the near-original quality of uncompressed baselines.

Model Files

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp