ddark-il/Qwen3.8-27B-Uncensored — MLX mixed-precision 5.59 bpw

Mixed-precision affine quantization of orcarouter/Qwen3.8-27B-Uncensored, a 64-layer hybrid trunk (48 Gated-DeltaNet + 16 softmax-attention) with an MTP draft head and a vision tower.

18.085 GiB · 5.592 bpw · 27.781 B params · 4 shards · 2.86× smaller than bf16

This revision replaces the previous files in place

The AWQ 5.0 bpw gs64 build previously published at this repo (16.161 GiB, 5 shards, revision bd19ce3eb9) has been replaced, not published alongside. Two consequences worth reading before you pull:

• Re-download rather than resume. The shard count changes 5 → 4, config.json changes, and the weight index changes, so a partially cached copy can end up with orphaned or mismatched files. If you need the AWQ build, it is still reachable by pinning the previous revision, ddark-il/Qwen3.8-27B-Uncensored@bd19ce3eb9210425915d18066ffc38478fb1b151 — it is simply no longer what the default branch serves, so pin a revision if you depend on it.
• Size and bit map both change: 16.161 → 18.085 GiB (+11.9 %), and AWQ's min/max ranges give way to a per-group range search plus 71 width upgrades. The loader contract is unchanged (__metadata__ = {"format": "mlx"}, same model_type, same tokenizer/chat template), so code and configs that loaded the old build load this one.

This build starts from the previous build's bit map, re-quantized from bf16 with a range search rather than AWQ, and then adds the 71 measured width upgrades.

Measured against the build this revision replaces

Paired per-position NLL against the bf16 source, 341,849 positions over two independent text sets (C headers; and news/wiki/prose/code), blocks resampled within each set:

• 30.0 % of the AWQ build's excess error over bf16 removed, pooled; 27.8 % on the C-header set and 33.8 % on news/wiki/prose/code.
• Better than the AWQ build by +0.00712 nats/token pooled [+0.00570, +0.00853], p = 3.3e-4.
• It is the only configuration measured that is better in all five domains by point estimate — but significant in four of the five: code2 is +0.00210 with CI [−0.00378, +0.00663], which includes zero. (An earlier draft of this card said "beats it in all five domains"; the per-domain check says otherwise, so it now says this.)
• Against the out_proj-only build (25.2 %) the margin is real but not decisive: p = 0.086.
• Re-quantizing the AWQ build's own bit map with this quantizer (no width changes, no added bytes) is also better than it — +0.00244 nats/token, p = 0.0073 by the paired bootstrap, positive in both text sets. This one is marginal and should be read that way: the same bootstrap's interval reaches −0.00009 at its lower bound, so the significance depends on which interval convention is used. The direction replicated across sets; the margin did not grow.

The bit map

589 modules are quantized; 442 tensors are copied bit-identically from the bf16 source, and 168 RMSNorm weights are stored with the unit offset folded in — bf16_round(1 + w) where the source stores w — which is the convention this model family's runtime expects (0 tensors are unexplained). Per-group affine ranges are chosen by a best-of {min/max, runtime edge-factor, bf16-scored uniform grid} search — scored in the format that is actually stored, since MLX keeps scales/biases in bf16 and a grid that wins in float32 can lose once cast.

Requires oMLX

Same requirement as the build this replaces, and unchanged by the re-quantization:

• Load it with oMLX. Stock mlx_lm/mlx_vlm either drop the MTP head and vision tower or mis-apply the norm convention described above. Every number in this card was produced through oMLX's own runtime.
• The MTP head ships inside the checkpoint (31 tensors under mtp.*) so the head can bind; enable it with the engine-level mtp_enabled setting for native multi-token-prediction decode. Its accuracy on this build is measured — see the limits section below.
• 27 vision linear_fc2 weights stay bf16 ([1152, 4304], input dim 4304 = 16 × 269 is not divisible by any supported group size), so the vision MLP is not quantized there. The merger's linear_fc2 (input dim 1152) is quantized.
• Tokenizer, chat template and configs are the fine-tune's own files, carried through unchanged.

Where the gain is concentrated, and why it is not a general improvement

The gain is not distributed evenly: out_proj/o_proj carries the most pooled (25.2 %), while on the code domain the pair embed_tokens + lm_head supplies 17.6 % of the AWQ build's excess there. Separating that pair with single-module builds (each byte-exactly one module different from the base) gives, on the code domain: embed_tokens removes 16.89 % of the AWQ build's excess there, lm_head 1.19 %. So the embedding is the contributor and lm_head is not — but two honest qualifications attach:

• That ordering is not statistically certified. embed_only vs base is p = 0.38 while head_only vs base is p = 0.046, because embed's gain is heavy-tailed: it wins on ~47 % of positions and wins large when it does. A direct embed_only vs head_only comparison is p = 0.48. It is a magnitudes reading.
• The effect is a small residual of two large sums. Gross gains +1,975 nats against gross losses −1,626 nats leave a net +349 nats, half of it from 29 of 184,230 positions. Only 6 positions in the whole domain are unambiguous embedding damage (base > 5 nats while bf16 < 1 nat); there the embedding recovers 73.8 % of the gap, which confirms the mechanism on clean examples — but that is 12.5 % of the effect. 37 % sits where bf16 also fails. Interpretable as rare-token embedding damage: the positions repaired are identifiers like self, raise, try, threading, snapshot.

Honest limits

Read these before relying on the model; they are the parts that are not verified.

• Fidelity is teacher-forced, not generative. Every number above is per-position NLL on held-out text. There is no long-horizon generation eval, no benchmark suite, no human evaluation. A build that matches bf16 in NLL can still drift over hundreds of generated tokens.

• Absolute tokens/sec from this project is a kernel-absent number. The fused GDN decode routes are keyed on class identity — this export's model_type: qwen3_5 resolves to a different module class than the fused kernels accept, and the eligibility predicate string-compares the class name. No config field can change that, and the recipe opt-in that widens the bit-recipe allow-list does not help either (this checkpoint's hidden size, 5120, fails the predicate's hardcoded 2560 geometry). Relative A/B timings are valid; absolute throughput is not measured here.

• The mtp draft head is measured — and in this build it is the better one, but not lossless. All main-model measurements run mtp=False (0 of 1,246 modules on the path), so the head's effect on the main model is unmeasurable. Its own job is measurable, and was measured directly: feed it the backbone's pre-final-norm hidden at position t plus the embedding of token t+1, and score its prediction of token t+2 (65,472 positions; bf16 is the oracle and is lowest, which is the check that the probe is sane):

  draft head	NLL (lower better)	top-1 agreement (acceptance proxy)
  bf16 oracle	1.92687	0.6253
  this build	1.93822	0.6239
  AWQ build's bit map (the base map)	1.94661	0.6223

  So the draft head is degraded only slightly versus bf16 (+0.0113 nats, −0.0014 top-1), and this build's mtp allocation is better than the AWQ build's by 0.0084 nats and +0.0016 top-1 (p = 0.0005 and p = 0.012). The 24.5 MiB spent on mtp buys a real improvement rather than being dead weight, and there is no measured reason to ship mtp_enabled=false on this build's account.

  One caveat on the absolute numbers: the head was fed the hidden from the ordinary forward, captured by wrapping the final RMSNorm, because the runtime's own return_hidden=True route forces speculative_verify=True and costs 330 s per 2,048-token block (~75× the ordinary forward). The two agree to r = 0.999738 (mean relative difference 2.6 %, identical scale and norm), so the between-arm comparison is fair — every arm used the same route — but a live draft cycle inside a verify pass may differ slightly, and top-1 agreement here is a proxy for acceptance rate, not a measured acceptance rate.

• L1 weight-error figures are over a sampled subset. The rel-L2 numbers used for per-group attribution cover ~4.6 B of the 27.64 B quantized elements (the verify harness reports the sample size it used). They are comparable to each other, not whole-model figures.

• Five text domains. Code (two corpora), prose, news, wiki, PTB. Nothing here characterises behaviour on chat, tool use, long-context retrieval, or languages outside English code/prose.

Files

config.json, generation_config.json, quant_log.json, chat_template.jinja, tokenizer.json, vocab.json, tokenizer_config.json, merges.txt, preprocessor_config.json, video_preprocessor_config.json, LICENSE, model.safetensors.index.json, and 4 safetensors shards (__metadata__ = {"format": "mlx"}). Use with MLX / oMLX; the artifact was verified to load and generate under omlx serve with the same loader contract the AWQ build had, so it is a drop-in replacement.

Verification

Reproducible from the bf16 source plus a JSON bit map: the export is deterministic (two independent export runs produced byte-identical per-class error, and byte_diff confirms each ablation build differs from its base in exactly the intended modules and nothing else). The measurement harness — evaluated per-position NLL, the block-clustered paired bootstrap, the ablation maps, and the raw result JSONs — is kept alongside the maps so every figure above can be re-derived rather than re-read.