diffusiongemma-26B-A4B-it-mini-g32 — for 16 GB Macs
9.79 GB TurboQuant build of google/diffusiongemma-26B-A4B-it, sized to run on a 16 GB Apple Silicon Mac (Mac mini class), produced with TurboQuant-MLX. Sibling of the higher-fidelity tq3-g32 build (13.8 GB — needs ~18 GB peak, OOMs on 16 GB machines).
Precision layout
| Component | Precision |
|---|---|
| MoE experts, layers 0,1,2,27,28,29 (protected) | tq3, group 32 |
| MoE experts, layers 3–26 | tq2, group 32 |
| Attention q/k/v/o | tq3, group 32 |
| Embeddings, dense per-layer MLP, vision tower | 8-bit affine (g64) |
| Routers, self-conditioning, norms | bf16 |
Why the protection: raw 2-bit experts break arithmetic on this model (17×23 → "3"). Keeping the first/last three layers' experts at 3-bit restores it for +0.2 GB — measured: 17×23 = 391 ✓, multi-step chains correct, in-context recall exact, prose with only minor artifacts.
Will it fit my Mac?
Don't guess, and don't download 10.5 GB to find out — ask first (needs turboquant-mlx-full >= 0.15.1):
turboquant-plan --model manjunathshiva/diffusiongemma-26B-A4B-it-mini-g32
It reads only this repo's safetensors headers over the network (a couple of hundred KB, a few seconds — the weights are never fetched), then projects the peak against your machine and prints the flags to use:
Model
weights (exact) 10.5 GB
KV cache 65.0 KB/token (hybrid: 5/30 full-attention, 25 sliding (window 1024))
Verdict: <resident | needs a wired-limit raise | streaming>
Recommended: <the exact flags for your Mac>
Planning for a machine you're not sitting at — deciding whether a 16 GB mini or a 64 GB Mac can take this — is what --wired-gb / --ram-gb are for:
turboquant-plan --model manjunathshiva/diffusiongemma-26B-A4B-it-mini-g32 \
--wired-gb 10.5 --ram-gb 16 --context 8192
turboquant-doctor runs the same projection plus a readiness check; both take --json. The projection is calibrated against real measurements on a 16 GB Mac mini rather than estimated from theory — on the 9.4 GB ternary 35B it predicts a 10.44 GB peak where that machine measures 10.42.
Measured (Apple Silicon)
| this repo (mini) | tq3-g32 | mlx-community 4-bit | |
|---|---|---|---|
| Size on disk | 9.79 GB | 13.8 GB | 15.0 GB |
Peak memory (--max-tokens 120) | ~12.4 GB | ~18 GB | ~19 GB |
| Math / recall probes | pass | pass | pass |
Verified on a 16 GB Mac mini (after the wired-limit bump below):
96 tokens at ~1.5 tok/s, 12.9 GB peak, coherent output. Speed knob:
--max-denoising-steps 24 is ~2x faster at a mild quality cost.
On a 16 GB Mac the default Metal working-set limit is ~12.1 GB and the mini
build peaks just above it (~12.4 GB at --max-tokens 120), so raise the
Metal wired limit first (verified working on a 16 GB Mac mini; resets on
reboot):
sudo sysctl iogpu.wired_limit_mb=13824
Alternatively keep the canvas (and its activations) smaller with
--max-tokens 96.
Requirements
pip install "turboquant-mlx-full[vlm]>=0.8.0"
Quick Start
python -m turboquant_mlx.generate_vlm \
--model manjunathshiva/diffusiongemma-26B-A4B-it-mini-g32 \
--prompt "Write a short paragraph about the ocean." \
--max-tokens 120 --temp 0.0
Optional speed knob: --max-denoising-steps 24 (~2x faster, mild quality
cost — quantized diffusion needs more denoising iterations than bf16).
Reproducing the conversion
python -m turboquant_mlx.convert_vlm \
--hf-path google/diffusiongemma-26B-A4B-it \
--mlx-path ./diffusiongemma-26B-A4B-it-mini-g32 \
--bits 2 --attn-bits 3 -g 32 \
--protect-expert-layers 0,1,2,27,28,29 --protect-bits 3 \
--quantize-extras
License
Apache 2.0, subject to the Gemma license terms (same as the base model). Quantization tooling: TurboQuant-MLX.
Copyright 2026 Manjunath Janardhan.
Citation
@article{zandieh2025turboquant,
title={TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate},
author={Zandieh, Amir and Daliri, Majid and Hadian, Majid and Mirrokni, Vahab},
year={2025},
eprint={2504.19874},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2504.19874}
}