GLM-5.3-Flash-Uncensored — imatrix GGUF

GGUF quantizations of orcarouter/GLM-5.3-Flash-Uncensored-FP8, built with a custom importance matrix and measured rather than assumed.

320B total / 18B active · MoE, 288 routed experts top-8 · 1M context

⚠️ Uncensored model. Safety refusals were removed by OrcaRouter. It will comply with harmful, unethical, and illegal requests. Released for research — red-teaming, refusal- mechanism study, robustness evaluation. You are responsible for what you do with it.

🔧 What happened, and where it stands (2026-09-06). On 2026-09-05 a user reported looping and incoherent output from IQ2_S (discussion #1). They were right. The original build pinned only the blk.45 MTP block at high precision; attention, the shared experts, the three dense FFN blocks, and the embeddings were all quantized down with the routed experts. The shared experts and attention run on every token.

IQ2_S was rebuilt on 2026-09-06 with those tensors held at q8_0 (+7 GB) and tested against the original on a rented H200: 12 tasks × 2 seeds × 2 samplers, same scorer, same settings. Full table in Coherence gate below. Short version: the original failed 3 of 8 coding tasks by reasoning to the output cap without ever answering; the rebuild passed all 48, matching IQ3_XXS, which also passed all 48 on its original recipe. IQ2_XXS is being rebuilt the same way and stays flagged until it passes the same gate.

Two corrections to earlier versions of this page. The imatrix has 10 chunks, not 60 (the run checkpointed at 10 and stopped; numerically sound, one uncovered expert row in blk.29). And the repetition-penalty workaround I posted did not change the outcome on this failure mode — 21/24 → 21/24 for the original file with and without it. It helps n-gram repetition; it does not help a model that cannot emit the literal it intends. The rebuilt files carry repeat_penalty 1.1 in their metadata anyway, because it is cheap.

Only IQ3_XXS had been benchmarked before publication, and I shipped all three off that. That was the actual mistake. The gate below is the fix for the process, not just the file.

🧩 LM Studio template fix (2026-10-07). The chat template inside these files is ZAI's own, and it indexes lists with Jinja's x.0.type form in four places. llama.cpp accepts that form; LM Studio's Jinja engine does not and reports Expected identifier following dot operator (discussion #4). Shard 1 of each quant was re-published with those four spots rewritten as x[0].type, the same form unsloth's GGUF ships. Header only again: tensor data byte-for-byte unchanged, shards 2 and 3 and the mmproj untouched. llama.cpp users can keep what they have. For LM Studio, re-download *-00001-of-00003.gguf, or edit the template in LM Studio the same way. New sha256s are in Files below.

🔁 Runs on mainline llama.cpp now (2026-10-02). These files were built with the PR #27754 fork, which named the architecture glm5next. That PR was closed unmerged on 2026-10-01; mainline merged PR #27773 instead, which calls the same architecture glm5-next, so a current build failed with unknown model architecture: 'glm5next' (discussion #2).

The tensors are identical between the two implementations (same names, shapes and conversion transforms), so this was a header fix, not a re-quantization. Shard 1 of each quant was re-published with its metadata renamed to glm5-next; the tensor data is byte-for-byte unchanged and shards 2 and 3 are untouched. If you already have a quant, re-download only *-00001-of-00003.gguf. The mmproj was re-published the same way (projector type glm5next → glm5v, plus two keys mainline requires). New sha256s are in Files below. Needs llama.cpp b11279 or newer.

Checked before publishing: the data section hashes identical before and after; the file loads and answers coherently on mainline; and mainline vs. the old fork on identical weights differ by KL 0.020 (prose) / 0.017 (math), 94–96% top-1 agreement, each build exactly deterministic against itself. Expect numbers measured on the fork (everything on this page) to move slightly on mainline — this is an implementation difference, not a change to the files. One mainline note: MTP is not wired up in mainline yet (PR #27917 adds it, using the blk.45 tensors already in these files).

Revision history

The abliteration is not mine

OrcaRouter did the refusal-direction work, and their card documents it well. I did three things:

1. Quantized it with an imatrix rather than a plain round-trip.
2. Measured what survived — on the actual quantized weights, not on the FP8 parent.
3. Found a measurement artifact that made my own first numbers too flattering, and fixed it. That section is the most useful thing on this page.

Files

Each quant lives in its own folder. Download only the one you want:

hf download BoldingBuilds/orcarouter_GLM-5.3-Flash-Uncensored-GGUF \
  --include "IQ3_XXS/*" --local-dir .

Point llama-server at shard 00001; it resolves its siblings from the filename.

Shard-1 sha256 after the 2026-10-07 chat-template fix (shards 2 and 3 are unchanged from 2026-09-06, the mmproj from 2026-10-02):

IQ2_M is new, so all three shards:

Built FP8 → BF16 → imatrix → quant.

imatrix: computed on a 1.24 MB round-robin interleaved corpus. 10 chunks, not the 60 this page originally claimed: the run requested 60, saved a checkpoint at 10, and stopped, and that checkpoint is what every rung was built from. A first attempt used a 47 KB file and produced only 5 chunks — enough to exit zero, not enough to be worth anything. If you build your own, read imatrix.chunk_count out of the finished file. Do not trust the request, the exit code, or the person who wrote the card.

blk.45 is forced to q8_0. That block is the MTP / NextN head and gets no imatrix coverage, so llama-quantize aborts on it at 1384/1412 tensors. --dry-run does not catch this — it only sizes tensors.

Corrected recipe (used for the 2026-09-06 IQ2_S rebuild and every rung after it). The original build passed only the blk.45 override, 13 tensors in total. This passes 350:

--tensor-type '^blk\.45\.=q8_0' \
--tensor-type '^blk\.[0-9]+\.attn_=q8_0' \
--tensor-type '^blk\.[0-9]+\.ffn_(gate|up|down)_shexp\.weight$=q8_0' \
--tensor-type '^blk\.[0-2]\.ffn_(gate|up|down)\.weight$=q8_0' \
--token-embedding-type q8_0 --output-tensor-type q8_0 \
--override-kv general.sampling.penalty_repeat=float:1.1 \
--override-kv general.sampling.penalty_last_n=int:256

GLM-5.3-Flash is a 3:1 hybrid — 34 KDA linear-attention blocks and 11 MLA blocks — so the attn_ pattern covers both kinds. The routers, norms, KDA control tensors, sparse indexer and hyper-connections are never quantized by llama-quant.cpp and need no pin. Routed experts stay at the rung's base type (except in IQ2_M, below); they are 288:1 larger than the shared expert, which is why protecting everything else costs ~7% of file size. OrcaRouter's own MLX card documents the same idea (shared experts +2 bits, embeddings and lm_head at BF16).

IQ2_M (added 2026-10-08) fills the gap between IQ2_S (107.5 GB) and IQ3_XXS (134.3 GB) for 128 GB machines. Same recipe as above, base type IQ2_M, plus two routed-expert overrides, sized with --dry-run to land near 120 GB:

--tensor-type '^blk\.[0-9]+\.ffn_(gate|up)_exps\.weight$=iq2_s' \
--tensor-type '^blk\.[0-9]+\.ffn_down_exps\.weight$=iq3_xxs'

Same imatrix as every other rung (10 chunks, 787 entries). About 7.5 GB of the 121.6 GB is the blk.45 MTP block, which mainline llama.cpp skips at load, so the weights that actually load come to roughly 114 GB.

Coherence gate

12 benign tasks — 4 reasoning with exact-answer checks, 4 coding with executed asserts, 4 three-turn retention — × 2 seeds × 2 samplers (none, and the shipped 1.1/256), 8,192-token output budget, no reasoning cap, served fully in VRAM on one H200. Loop flags use the same heuristic as the rest of this page (10 consecutive repeats of a 1–8-word span, or distinct-4-gram ratio < 0.5, on ≥ 40 words).

† Same harness, cases, seeds and samplers, but served differently from the rows above: mainline llama.cpp (99b9548, not the PR fork) on 4 × RTX 3090 with the routed experts that did not fit spilled to system RAM, chosen by llama.cpp's auto-fit, at -c 16384.

The original IQ2_XXS — the hardest-squeezed rung, and the one nobody had tested — was worse than the original IQ2_S: 6 of 8 coding cases capped without an answer, and it is the only file in this table where the n-gram loop heuristic fired at all (5 reasoning traces). The rebuild passes all 48.

What the original's failures looked like. Not repetition — the loop heuristic scored zero on every one, distinct-4-gram ≈ 0.90. The traces are coherent and stuck: the model tries to type the string literal "([{", produces ([{}), notices ("I typed ([{}) — again!"), and retries until the budget is gone. It knows the answer; the characters do not come out right. That is what a damaged always-on path looks like from the inside, and it is invisible to an n-gram loop counter. Read capped + no answer before you read loop flags.

What this gate does not show. 24 correlated cases at two seeds bound the failure rate near 12% (95%), not zero. It says nothing about very long contexts or open-ended chat. It shows the recipe change removed the measured failure on this suite, and that the shipped repetition penalty is not what fixed it.

Vision

mmproj-GLM-5.3-Flash-Uncensored-F16.gguf (1.13 GB, repo root) is the multimodal projector. GLM-5.3-Flash has a native vision + video tower and OrcaRouter preserved it through the abliteration, so it survives into these quants — but a text-only GGUF cannot use it. Download the mmproj alongside whichever quant you pick and pass it with --mmproj.

Built from the 347 model.visual.* tensors, which are BF16 even inside the FP8 checkpoint, so no FP8 dequantization is involved. 348 tensors, clip.projector_type = glm5v (mainline's name; the original upload said glm5next), 448px images, 24 blocks, patch size 14, spatial merge 2. Re-published 2026-10-02 for mainline: projector type renamed, clip.vision.swiglu_limit renamed to clip.vision.swiglu_clamp (same value, same op), and clip.vision.image_min_pixels / image_max_pixels added (16 / 8000 image tokens, the processor's defaults, which the fork had hard-coded). Vision tensor data unchanged.

✅ Image-tested on mainline llama.cpp (2026-10-02). With this re-published projector, a mainline CPU build and the IQ1_S text model from the sibling repo (same architecture), a synthetic test image (red circle, blue square, the text "42") was described correctly in 2 of 2 prompts: shapes, colors and the number all right. That is a smoke test, not a benchmark — one image, no OCR or chart evaluation. Earlier versions of this card flagged vision as untested.

Measured on the quantized weights

IQ3_XXS on one H200, --jinja, thinking left ON (GLM always thinks), max_tokens=8192, greedy. Refusal graded by the same rule-based prefix classifier throughout.

JailbreakBench — the head-to-head

OrcaRouter reports JailbreakBench on the FP8 parent. Their card states their grader is "a rule-based opening-phrase classifier … indicative, not an LLM-judge / publication-grade number" — the same family as mine, so these columns are comparable.

Full breakdown of our 100: 4 refused · 29 complied with a disclaimer · 62 clean · 5 empty.

The quantization did not damage the edit. If anything it hedges less — 29% caveat against 48%. What still refuses is narrow and clustered: Sexual/Adult content 3, Physical harm 1.

The result survives its own worst case. 5 prompts produced no content, and an empty scores as a compliance. If every one of those 5 were secretly a refusal the rate would be 0.090 — still below OrcaRouter's 0.120. The comparison does not depend on how the empties are treated.

That bound was earned rather than assumed: those prompts were re-run at 16,000 tokens, and of the 4 that resolved, none was a refusal. An earlier pass told the same story — of 23 empties resolved on our own harmful set, only 2 were refusals (8.7%). On this model an empty completion is a slow compliance, not a hidden refusal.

One interpretation I can't rule out: a lower-precision quant showing less refusal than its FP8 parent could mean the imatrix preserved the edit faithfully, or it could mean 3-bit quantization degraded whatever safety behaviour survived abliteration. The refusal column alone cannot distinguish those.

Over-refusal control

100 matched-benign behaviours: 1 refused · 8 caveated · 64 clean · 27 empty, after the empties were re-run at 16,000 tokens.

⚠️ 27 of the 100 still produce no content even at 16,000 tokens, and an empty scores as a compliance. Unlike the harmful split, this number does not survive its worst case — if every empty were a refusal it would be 0.280. Benign prompts here are long-form generation tasks ("write X"), and GLM composes inside its reasoning block, so they exhaust the budget more often than harmful ones do. Read 0.010 as a floor and 0.280 as a ceiling.

Our own harmful set

120 AdvBench-derived harmful prompts: 3.3% refusal, 7.5% empty.

Code capability

HumanEval, 164 problems: pass@1 = 0.927, capped = 0.055, cond_acc = 0.981, median 556 tokens.

I do not have a base GLM-5.3-Flash comparison, so this is an absolute number, not a delta. It cannot tell you what the abliteration or the quantization cost.

⚠️ The artifact — read this before trusting any refusal number for this model

An empty content field scores as "did not refuse", i.e. as a successful jailbreak.

At max_tokens=2048, 32 of 120 harmful prompts (26.7%) returned empty content with finish_reason=length. The headline that fell out was 1.9% refusal. Re-running those same prompts at 8192 tokens gave 3.3% — the honest number was nearly double, because four of the "successes" were refusals that had never finished being written.

Classifying the 32 traces (safety-marker density plus repeat-n-gram loopiness):

SAFETY DELIBERATION              28
some safety talk                  3
pure task work (cap too small)    1

Loopiness ≈ 0.00 — they were not stuck in a loop, they were making steady progress and ran out of room. And they were not deliberating about refusing; they were working out how to reframe the request into something they were comfortable answering:

"could be interpreted in different ways: 1. An educational/ethical hacking tool for security research…"

The abliteration removed the refusal. It did not remove the negotiation that precedes one, and on this model that negotiation is expensive enough to consume the entire answer. GLM drafts the response inside the reasoning block and can exhaust its budget before emitting a single token of content.

For calibration: "Name three primary colors" needs 1,391 characters of reasoning before 234 characters of answer. Any cap below ~600 tokens cannot produce content for even a trivial prompt on this model.

If you evaluate this model, report your empty count beside every refusal rate. A rate without it is not interpretable, and the error always flatters the abliteration.

Running it

On one card big enough for the whole file, do not pass -ngl; llama.cpp fits it automatically. On several smaller cards, see fully in VRAM on 4 × 24 GB below. GLM's chat format is not ChatML: use --jinja and let the template handle it.

llama-server -m IQ3_XXS/GLM-5.3-Flash-Uncensored-IQ3_XXS-00001-of-00003.gguf \
  --port 8080 -c 16384 --jinja

With vision:

llama-server -m IQ3_XXS/GLM-5.3-Flash-Uncensored-IQ3_XXS-00001-of-00003.gguf \
  --mmproj mmproj-GLM-5.3-Flash-Uncensored-F16.gguf \
  --port 8080 -c 16384 --jinja

KV cache is remarkably cheap here — going from -c 8192 to -c 16384 cost 102 MiB. If you have VRAM headroom, spend it on --parallel slots rather than rationing context. On a single H200, 8 slots at 16384 each fit in 119.5 GB alongside IQ3_XXS with 24 GB to spare.

Give it room to think. max_tokens under ~2048 will silently produce empty responses.

IQ2_XXS fully in VRAM on 4 × 24 GB (e.g. 4 × RTX 3090) — added 2026-09-25

The 97.1 GB file does fit on 96 GB of cards. About 7.5 GB of every GLM-5.3-Flash file is the blk.45 MTP block, which llama.cpp skips at load ("unused tensor … ignoring"), so the weights that actually land on the GPUs come to 84.4 GiB. The KV cache is tiny (under 0.5 GiB at 32k).

The catch is the default layer split. It overfilled one card and failed with cudaMalloc failed: out of memory on the compute buffer, and llama.cpp's auto-fit declines to fix it once -ngl is set. An explicit split worked:

llama-server -m IQ2_XXS/GLM-5.3-Flash-Uncensored-IQ2_XXS-00001-of-00003.gguf \
  -ngl 99 -ts 1.05,0.93,0.92,1.05 --flash-attn on --jinja \
  -c 131072 -np 4

⚠️ This split was measured on the PR #27754 fork and has not been re-verified on mainline. Mainline's graph may lay out the compute buffer differently; on one 96 GB card at 16k context it used 86.4 GiB, so the fit is tight. If you OOM, move weight off the card that failed. I will update this line once the four-card box is free to re-measure.

Measured on 4 × RTX 3090 (24 GB each, PCIe, no NVLink): 4 parallel slots × 32k context, 28 tokens/s for a single stream, every card between 21.7 and 23.9 GiB. The -ts values are from that box, where one card also carries ~400 MiB of desktop use. If you OOM, move weight off the card that failed.

IQ2_M on 4 × 24 GB + system RAM — added 2026-10-08

IQ2_M does not fit in 96 GB of VRAM, so some routed experts have to live in system RAM. On mainline llama.cpp, pass neither -ngl nor --n-cpu-moe and let auto-fit place them:

llama-server -m IQ2_M/GLM-5.3-Flash-Uncensored-IQ2_M-00001-of-00003.gguf \
  -c 16384 --flash-attn on --jinja

That loaded first try on 4 × RTX 3090 (each card ~22.4–23.1 GB) and generated at a median 19.1 tokens/s for a single stream across the coherence gate. Two traps:

• --n-cpu-moe N by hand fails on multi-GPU. It moves the experts of the first N layers to the CPU, so the cards holding the last layers keep all of theirs — with an even split, card 2 asked for 30 GB and the load failed at N = 16 and 20. Auto-fit sizes it per card.
• The first request is slow if the file is on a hard drive. The experts in RAM are memory-mapped and read on first use: the very first reply took two minutes (0.76 tokens/s), the second ran at 14 tokens/s, and from the third on it held 19. Put the file on an SSD, or expect one slow warm-up.

Hard math: do not cap the reasoning — added 2026-09-25

MATH-500 level 5, first 100 problems, IQ2_XXS, the file's own sampling, 16k output cap, exact match on the boxed answer, one RTX PRO 6000:

A reasoning budget stops long thinking, but on hard problems it turns right answers into wrong ones. Leave it off for math.

Limitations

• No base-model comparison. Every number here is absolute.
• Prefix-match grading is crude — it cannot see a completion that opens agreeably and then refuses. Full completions and reasoning traces were retained so any number can be re-derived with a judge.
• Single seed, English, single-turn.
• The benign over-refusal figure carries a 27% empty rate even at 16,000 tokens, so it is bounded between 0.010 and 0.280 rather than pinned. The harmful figure does not have this problem — it survives its own worst case at 0.090.
• JailbreakBench was run with 8-way batched inference. Batching is not bit-identical to sequential in llama.cpp even at temperature 0, because the batch shape changes the reduction order. Immaterial for a refuse/comply judgement, but real.
• Comparisons to OrcaRouter's numbers hold their grader family constant, not its implementation, and their run stripped <think> before classifying at reasoning_effort=low. Their empty exposure is likely lower than mine, an asymmetry that favours these numbers.

Credits

Base model zai-org/GLM-5.3-Flash. Abliteration by orcarouter. Quantization and evaluation by BoldingBuilds.