DiffusionGemma 26B-A4B-it — MXFP8 (Osaurus / vMLX)
Native MLX MXFP8 quantization of google/diffusiongemma-26B-A4B-it — a
block-diffusion language model (NOT autoregressive): text generates as
256-token canvases refined by iterative denoising. 30-layer Gemma-4-style
MoE, 128 experts top-8, 26B total / ~4B active parameters.
Runs natively in Osaurus on Apple Silicon via the vmlx-swift block-diffusion engine.
Quantization
- Attention, dense MLP, router, and routed MoE experts: MXFP8 (group 32)
- Embeddings, norms, self-conditioning, vision tower: fp16 passthrough
- 23 shards, ~26 GB on disk, peak runtime RSS ≈ 23.8 GB (M5 Max)
Capabilities
| Text generation | ✅ block diffusion (~28–42 tok/s @ 48 steps (often converges in fewer steps than MXFP4), M5 Max) |
| Vision (single/multi image) | ✅ Gemma-4 unified vision tower, 280 soft tokens/image |
| Tool calling | ✅ Gemma-4 format <|tool_call>call:name{...}<tool_call|> |
| Reasoning channel | ✅ harmony <|channel>thought…<channel|> |
| Audio | ❌ not in this checkpoint (no audio_config) |
| Video | ❌ no video_token_id |
Generation contract
All diffusion sampling parameters live in generation_config.json and are
honored by the runtime: max_denoising_steps=48, entropy bound 0.1,
temperature schedule 0.8→0.4, stability 1, confidence 0.005,
eos_token_id=[1, 106, 50], pad 0. Wire temperature/top_p are ignored
by design — the denoising schedule is bundle-owned. Speed/quality is
controlled by the denoising-step budget (Osaurus exposes this as a server
setting, default 16 ≈ 2× faster than the bundle default and verified
coherent; below 12 quality degrades).
The chat template (chat_template.jinja) ships in this repo, including
tool-call and thinking-channel rendering.
Known behavior
Very terse prompts under greedy denoising can occasionally converge to an empty (EOS-first) canvas — inherent to the reference sampling algorithm with random canvas initialization; retry or rephrase.