GLM-5.3-Flash-Alis-MLX-6bit
MLX (mlx-vlm tree) 6-bit build of GLM-5.3-Flash
(320B-A18B, glm5_next), converted by streaming dequant of the official FP8 release (84c6a6aa).
This replaces the withdrawn earlier build (which quantized the MoE router and decoded at
≈5.5 tok/s). This build keeps the router (mlp.gate) and correction bias unquantized/fp32,
the mHC arrays and KDA A_log/dt_bias in fp32 as stored, and the vision tower in bf16;
the MTP layer (45) is dropped (a standalone MTP drafter is published separately as
avlp12/GLM-5.3-Flash-Alis-MTP-Drafter). Conversion receipts (state.json,
finalize_receipt.json) are in-repo.
Recipe
6-bit g64 affine on experts, attention, dense/shared MLPs, embeddings and head; router / mHC /
KDA decay params / norms / convs as stored; vision bf16. Per-module map in config.json.
Measured (M3 Ultra 512GB)
| prompt | decode tok/s (oMLX SSE) |
|---|---|
| p512 | 26.3 |
| p2048 | 25.3 |
| p4096 | 24.6 |
Held-out paired KL vs the 8-bit teacher of this family (en/ko/code, ctx-2048, same harness as
the 4-bit builds): 0.0269 / 0.0226 / 0.0177, weighted mean ≈ 0.0226 — versus 0.0569 for the
4-bit QUASAR build and 0.0622 for 4-bit RTN. Same comparability caveat as the 4-bit card:
quantized-teacher, private panel — not comparable to BF16-teacher KLD numbers elsewhere.
Add MLX_MAX_MB_PER_BUFFER=2048 MLX_MAX_OPS_PER_BUFFER=100000 to the serving environment for
≈+12% decode on M3 Ultra.
The fused-KDA Metal kernel and long-context gathered-prefill findings documented on the
4bit card (+20% decode, +50% prefill
at 131k, both bit-identical) are architecture-level and apply to this build too; measured on the
4-bit tree.
For the fastest 4-bit-class build of this family (QUASAR-init, held-out KL −8.5% vs 4-bit RTN),
see avlp12/GLM-5.3-Flash-Alis-MLX-4bit.
Runtime
Requires mlx-vlm with glm5_next (merged 2026-08-26); the text-stack correctness fixes
(swiglu clamp, fp32 mHC/KDA aux, eps) live in the PR
#2044/#2074
branches — recommended until merged.