GLM-5.3-Flash-MLX-oQ2-MTP

[!IMPORTANT] Baseline text and image generation pass. Native MTP at depth 1 is experimental in the tested runtime: it produced coherent output and higher throughput, but it was not token-identical to baseline across a 512-token greedy run. Keep MTP disabled when exact deterministic reproducibility matters.

At a glance

oQ recipe

The planner selected 555 Q8 boosts. The saved checkpoint has 554 Q8 overrides because language_model.mtp.0.eh_proj remains BF16 to protect MTP fusion. This accounts for the one-module difference.

All 36 sparse indexer projections are in the saved Q8 set, including the three native-MTP indexer projections. Every quantised module has matching weight, scale, and bias entries.

This is weight-only post-training quantisation. It does not retrain or fine-tune the upstream model.

Runtime compatibility

GLM-5.3-Flash uses the glm5_next multimodal architecture, hybrid linear and sparse attention, manifold-constrained hyper-connections, and one native next-token prediction layer. The oQ layout also needs per-module quantisation metadata support.

Use these versions for the reported results. Other versions may not understand this checkpoint's glm5_next, oQ, or native-MTP paths.

Download and use

hf download Vontra/GLM-5.3-Flash-MLX-oQ2-MTP \
  --local-dir ./GLM-5.3-Flash-MLX-oQ2-MTP

Add the downloaded directory to a compatible oMLX model directory and refresh the model registry. Start with native MTP disabled. If you choose to test MTP, use draft depth 1 and read the validation note below.

Apple M3 Studio performance

The baseline median comes from three 512-token runs. A separate clean-load pair used the same prompt and settings once per mode to compare MTP directly.

The matched MTP run was 7.1% faster. It is one run, not a median. Both modes generated 512 coherent tokens, but their exact greedy token sequences first differed at zero-based output index 313. Treat the speed result as experimental rather than a bit-exact acceleration claim.

Architecture

GLM-5.3-Flash combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, and one native next-token prediction layer.

See the official model card, Z.ai overview, and GLM-5 technical report for upstream training, evaluations, intended uses, and safety guidance.

Validation

Limitations

• Mixed-precision quantisation can reduce quality relative to the official checkpoint. The effect depends on the workload and allocation.
• Proxy-derived sensitivity scores depend on the calibration protocol and do not guarantee BF16-equivalent output.
• Native MTP helps only when draft acceptance offsets drafting and verification cost. It may be neutral or slower for unpredictable output.
• The current native-MTP throughput result is one matched run. Compare more runs before drawing a firm performance conclusion.
• In the tested runtime, native MTP was not token-identical to baseline over 512 greedy output tokens. Leave it disabled for bit-exact deterministic workflows.
• The configured one-million-token context does not mean every Apple silicon system has enough memory for a full-context request.
• Image and video prefill have different memory and throughput characteristics from text-only generation.
• Runtime support for glm5_next, oQ metadata, and native MTP is version-sensitive.

This is a community quantisation, not an official Z.ai release.

Licence and attribution

The upstream model uses the MIT License. The official licence text is included as LICENSE.

Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The oQ conversion, native-MTP preservation, validation, and packaging are provided by Vontra.

If you use this model in research, cite the upstream report:

@misc{glm5team2026glm5,
  title        = {GLM-5: from Vibe Coding to Agentic Engineering},
  author       = {GLM-5-Team and others},
  year         = {2026},
  eprint       = {2602.15763},
  archivePrefix= {arXiv},
  primaryClass = {cs.LG},
  url          = {https://arxiv.org/abs/2602.15763}
}

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

Quick start and demo prompt

hf download Vontra/GLM-5.3-Flash-MLX-oQ2-MTP --local-dir ./models/GLM-5.3-Flash-MLX-oQ2-MTP

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow Vontra for new Apple Silicon releases and fixes.