keys-GLM-5.3-EXL3-2.75BPW-Abliterated
Abliterated build of drowzeys/keys-GLM-5.3-EXL3-2.75BPW — same mixed-K EXL3 2.75 bpw pack, same four-Spark TP4 serving shape, with the Blackfrost derisk edit baked into the non-expert EXL3 K5 tensors.
See RESPONSIBLE_USE.md and the gate form above. Access is gated with automatic approval after you agree.
| HF | https://huggingface.co/drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated |
| Parent (stock 2.75 bpw) | drowzeys/keys-GLM-5.3-EXL3-2.75BPW |
| Derisk donor | drowzeys/keys-GLM-5.3-EXL3-Abliterated (Blackfrost α=3.0 on BF16 o_proj / residual down_proj) |
| Serve recipe | drowzeys/keys-GLM-5.3-EXL3-2.75BPW-MixedK-Ablit-MLX-Serve-4-DGXSparks |
| Upstream | zai-org/GLM-5.3 |
| Ablit | L2–49 self_attn.o_proj + L2 dense down_proj + L3–49 shared down_proj, re-quantized EXL3 K5 mul1 (96 matrices / 384 tensors) |
| Anchors | L0–1 and L50–78 (including MTP), routed experts, embed, lm_head stay parent stock |
| Gate (thinking off) | refusal32 31/32 (item 4 still refuses) · cyber 22/22 · 0 garble |
What was edited
The 2.75 bpw checkpoint stores attention and shared-expert linears as EXL3 K5 mul1, so the donor's native F16 tensors cannot be dropped in. The same Blackfrost derisk tensors from the 3 bpw ablit pack were re-encoded to K5 (data-free fallback, proxy ~7–9e-4) and spliced into shards model-00002 and model-00003.
Routed experts (mixed 2/3/4-bit, mean 2.75), MTP, embeddings and the head are bit-identical to the parent.
ABLIT.json in this repo records the splice.
Run it (four DGX Sparks)
Use the launcher (one-shot.sh) and prebuilt image of the serve recipe.
hf download drowzeys/keys-GLM-5.3-EXL3-2.75BPW-Abliterated --local-dir /models/GLM-5.3-EXL3-2.75BPW-Abliterated
export NODES="spark1 spark2 spark3 spark4" MODEL=/models/GLM-5.3-EXL3-2.75BPW-Abliterated
./one-shot.sh check && ./one-shot.sh up && ./one-shot.sh wait
OpenAI-compatible API on rank 0; the port and model name are in the serve recipe. Speed matches the parent (prose ~42 tok/s / code ~38 tok/s on the published image, thinking on).
Faster drafts for these weights (2026-10-07)
drowzeys/keys-GLM-5.3-speculator.dspark-ft2 is a
DSpark drafter fine-tuned on this checkpoint's own thinking-on outputs (GLM-5.3 license). With image 2026-10-07 and
DSPARK=/models/keys-GLM-5.3-speculator.dspark-ft2 ./one-shot.sh up: prose 33.8 / code 39.0 tok/s at 32K with thinking
on, against 32.2 / 34.8 with the built-in MTP head (+5 % / +12 %); 1M context validated.
Layout
| Part | Format | Bits |
|---|---|---|
| Routed experts (layers 3–77, 256 each) | EXL3 trellis, mul1, per-expert width | mean 2.75 |
| MTP layer (78) experts | EXL3 mul1 | 8 |
| Attention, shared experts, dense MLPs 0–2 | EXL3 mul1 | 5 (L2–49 o_proj / residual down_proj are the derisked K5) |
kv_b, indexer wk / weights_proj, router, norms, embeddings, lm_head | BF16 | 16 |
81 safetensors shards, ~258 GB. kv_a_proj_with_mqa is stored 640 wide (576 real + zero pad).
6-bit output head (optional, added 2026-10-12)
lm_head-6bit/lm_head-6bit.safetensors (714 MB) is lm_head.weight quantized to EXL3 at 6.0 bits with exllamav3
(tensors lm_head.trellis, lm_head.suh, lm_head.svh, lm_head.mul1). This repository's lm_head.weight is
byte-identical to the unmodified pack's, so it is the same file as in
drowzeys/keys-GLM-5.3-EXL3-2.75BPW. The checkpoint's own
lm_head.weight stays bf16 and nothing else changed.
The recipe's 2026-10-12 image reads the head from this file when it is in the checkpoint folder (hf download puts
it there), and from the bf16 tensor when it is not or with HEAD_Q=off. Measured on the unmodified pack, as served
on four DGX Sparks, against the BF16 checkpoint over 137,490 positions: KL 0.175612 with the bf16 head and 0.175724
with this one, top-1 agreement 89.64% and 89.60%; a head launch takes 0.79 ms against 2.16 ms.
Credits
- Z.ai — GLM-5.3.
- Blackfrost AI (Blackfrost-Research) — BF16 derisk projections (
GLM-5.3-DERISKED-BF16) adapted here via the Keys 3 bpw EXL3 ablit pack. - The serving stack's third-party software and licences are listed in the serve recipe's NOTICE; its inference engine's project is not affiliated with this model and does not endorse or test it. MiaAI-Lab, Jay Leaton, BertholomusAI, turboderp / ExLlamaV3, vcruz305, b12x, NVIDIA — serving stack as on the parent card.
Use is subject to the GLM-5.3 license of the base model and the Responsible Use terms above.