GLM-5.3-Flash-MLX-4bit-MTP

At a glance

Modules that are not eligible for MLX quantisation remain at their source-compatible precision. The checkpoint preserves the upstream tokenizer, chat template, multimodal processor, generation configuration, and native MTP metadata.

Quantisation recipe

This is a weight-only post-training quantisation. It does not retrain or fine-tune the upstream model.

Runtime compatibility

GLM-5.3-Flash uses the new glm5_next multimodal architecture, hybrid linear and sparse attention, manifold-constrained hyper-connections, and an embedded next-token prediction layer. All four features need explicit runtime support.

[!IMPORTANT] A runtime that can load glm5_next is not automatically able to use its embedded MTP layer. Native MTP requires the GLM5 Next-specific draft, verification, and cache-reconciliation path. If that integration is absent, keep MTP disabled or use a compatible build.

Download and use

hf download Vontra/GLM-5.3-Flash-MLX-4bit-MTP \
  --local-dir GLM-5.3-Flash-MLX-4bit-MTP

In a compatible oMLX build:

1. Add the downloaded model directory.
2. Keep Native MTP disabled for the currently validated serving path.
3. Start a new conversation after changing model settings so the runtime rebuilds the model and caches.

For baseline generation without native MTP, a compatible MLX-VLM installation can use:

python -m mlx_vlm.generate \
  --model Vontra/GLM-5.3-Flash-MLX-4bit-MTP \
  --prompt "Explain how hybrid attention reduces long-context serving cost." \
  --max-tokens 512

Apple M3 Studio performance

Measured on an Apple M3 Studio with a warmed model state:

These figures are a local reference, not a cross-platform guarantee. Prompt shape, image input, context growth, draft acceptance, sampling, memory pressure, thermal state, and runtime revisions can materially change performance.

Native MTP status

The checkpoint contains the official matching NextN layer. A short deterministic MTP smoke test produced the exact requested output and recorded an accepted draft. Sustained testing on the current oMLX build exposed repeated speculative state reinitialisation and lower throughput than baseline decode. Native MTP is therefore preserved for compatible future runtimes but is not recommended for sustained serving on the validated build. Keep MTP disabled unless your runtime has corrected that scheduler behaviour.

Architecture

GLM-5.3-Flash is Z.ai's first natively multimodal GLM-5 model. It combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, and a native next-token prediction layer.

See the official model card, Z.ai overview, and GLM-5 technical report for upstream training, evaluations, intended uses, and safety guidance.

Validation

Validation used oMLX 0.6.3rc3, MLX 0.32.0, and MLX-VLM 0.6.3 with GLM5 Next runtime integration.

Limitations

• Quantisation can reduce quality relative to the official checkpoint, especially on difficult reasoning, tool-use, multilingual, vision, and long-context workloads.
• Native MTP improves speed only when draft acceptance offsets drafting and verification cost. It may be neutral or slower for unpredictable output.
• The configured one-million-token context does not imply that every Apple-silicon system has enough memory for a full-context request.
• Image prefill has different memory and throughput characteristics from text-only generation and is not represented by the decode benchmark above.
• Runtime support for glm5_next and its native MTP path is evolving. Use the exact validated versions listed in this card.
• Validate numerical behaviour, output quality, safety, and tool use on representative workloads before production deployment.

This is a community quantisation and is not an official Z.ai release.

Licence and attribution

The upstream model is released under the MIT License. The required licence text is included in this repository.

Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The MLX conversion, Apple-silicon validation, native-MTP integration work, and packaging are provided by Vontra.

If you use this model in research, cite the upstream report:

@misc{glm5team2026glm5,
  title        = {GLM-5: from Vibe Coding to Agentic Engineering},
  author       = {GLM-5-Team and others},
  year         = {2026},
  eprint       = {2602.15763},
  archivePrefix= {arXiv},
  primaryClass = {cs.LG},
  url          = {https://arxiv.org/abs/2602.15763}
}

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

oMLX version mentioned in the existing card: 0.6.3rc3; consult its compatibility notes for whether this was tested and any required integrations. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

Quick start and demo prompt

hf download Vontra/GLM-5.3-Flash-MLX-4bit-MTP --local-dir ./models/GLM-5.3-Flash-MLX-4bit-MTP

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow Vontra for new Apple Silicon releases and fixes.