ToPo-ToPo/gemma-4-31b-it-mlx-bf16
MLX bf16 conversion of google/gemma-4-31b-it for Apple Silicon (mlx-vlm).
Provenance (self-converted from official weights)
- Source:
google/gemma-4-31b-it(license: gemma) - Tool:
mlx-vlm 0.6.3—mlx_vlm.convert --hf-path google/gemma-4-31b-it --mlx-path . --dtype bfloat16 - Effective: 16.0 bits/weight
- Validation: reproduced geometrically exact CAD output in an agentic CAD+FEM pipeline (volumes match the reference mlx-community conversion).
Usage
from mlx_vlm import load, generate
model, processor = load("ToPo-ToPo/gemma-4-31b-it-mlx-bf16")
License
This is a derivative of Google Gemma. Use is governed by the Gemma Terms of Use and the Gemma Prohibited Use Policy. Weights were converted/quantized to MLX format (modification notice per the Gemma Terms).
⚡ MTP drafter (speculative decoding)
Use google/gemma-4-31b-it-assistant — Google's official MTP drafter for
this model. It loads directly in mlx-vlm (>= 0.6.3), needs no conversion, and speculative
decoding is lossless. Drafters are size-specific and not interchangeable across Gemma 4
variants.
🔧 Patched chat template
chat_template.jinja differs from Google's Gemma 4 Canonical Chat Template
(2026-07-09) by one intentional change; everything else is untouched.
The canonical template suppresses the thinking channel at the start of a normal model
turn, but emits nothing after a tool response when enable_thinking is false. The model
may then open a thinking channel on its own, and a quantized model sometimes writes the
literal word thought into the answer. This patch gives the tool_response branch the
same suppression:
{%- elif ns.prev_message_type == 'tool_response' -%}
{%- if enable_thinking -%}
{{- '<|channel>thought\n' -}}
{%- else -%}
{{- '<|channel>thought\n<channel|>' -}}
{%- endif -%}
{%- endif -%}
Only that case changes — the other prompt paths render byte-identical to the canonical
template. To get stock behaviour, replace chat_template.jinja with the one from the
base model repo; the weights are unaffected.