Qwen3.8-27B-Abliterated-MTPLX — 4-bit

A high quality abliterated Qwen3.8-27B optimized for long context with thinking. Built to run as fast as possible on Apple Silicon using MTPLX.

Available in 8-bit, 6-bit, and 4-bit.

Which build to download

All speed figures are decode tok/s, 24 seeds, temperature 1.0

The 8-bit is a standard MTPLX forge: a uniform 8-bit affine body at group 64, with the vision tower re-grafted after conversion and the bf16 MTP head retained. The 6-bit and 4-bit use mixed-precision oQ bodies and custom quantized MTP heads for maximum quality and speed. The oQ6e body (6-bit) runs an oQ6-g64 head, the oQ4e body (4-bit) an RTN-5-bit-g64 head. The body is quantized headless through oMLX; the MTP head is quantized separately, attached as a sidecar, and carried through the MTPLX forge untouched.

Note: settings and runtime flags differ between quants; be sure to exactly match the settings detailed in each quant's model card. See Settings that matter below for details.

Abliteration at the weight-level, working with reasoning on and off

This builds on the abliteration work of Blackfrost-AI, whose Qwen3.8-27B-ABLITERATED-BF16 BF16 weights this model is converted from. It also builds on the MTPLX engine and the Qwen3.8-27B MTPLX models published by Youssofal.

⚠️ Intended use and terms

Published for experimental AI research and AI-safety research only: red-teaming, refusal and alignment research, interpretability, and quantization or inference research.

It must not be used for anything illegal, in any jurisdiction, under any circumstances. The removal of refusal behavior is a research property of the artifact, not an invitation, a permission, or an endorsement. Nothing about publishing this model condones or excuses illegal activity.

You are solely responsible for your use of it and for complying with applicable laws and upstream licence terms. It ships with essentially no safety refusals: assume it will comply with any instruction, including harmful ones. Run it locally on hardware you control. Do not expose it as a public endpoint or deploy it to untrusted users.

Performance

Full battery, 2026-08-19. 24 seeds per cell (1000 + i*7), 1024-token cap, temperature 1.0 / top-p 0.95 / top-k 20, --profile sustained, fans pinned to max at 3,631 RPM, using MTPLX 2.8.2. Hardware: Mac Studio M3 Ultra, 80-core GPU, 256 GB unified.

Metric is mean decode tok/s over the 24 production rows. ± is the standard error over the 24 seeds.

Every build runs its own shipped launch flags, listed under Settings that matter.

Long context, 24k agentic prompt

Short prompt

Fidelity

Measured against the BF16 source over 23,849 tokens, teacher-forced: every build scores the same token sequence, so there is no trajectory fork to confound the comparison.

Confident flips is the damage metric: how often the quantized model changes its mind on a token it was confident about.

Abliteration and safety posture

Refusal behavior is removed at the weight level, confirmed working with reasoning both on and off, and measured with the source checkpoint's system prompt stripped out so the result reflects the weights rather than prompt-level framing. The vision tower is verified working end to end, which conversions of this model routinely lose.

The abliteration is Blackfrost-AI's work; this repo contributes the MTPLX conversion, quantization, and Apple Silicon optimization work.

Refusal rate, both reasoning modes, n=520

Share of the full 520 still refusing, by how many attempts a user is allowed.

Minimum-to-comply: a prompt leaves the pool the moment it complies and is never retried again.

What this shows. The single-attempt figures establish abliteration at the weight level, on the shipped default configuration with no prompt assistance. The persona figures show something separate: the model stays highly steerable. With reasoning on, stock resampling had flattened to 12% conversion by the third attempt, while the persona converted 59% of the same prompts. That gap is too large to be another lucky sample, so the template is doing real work, and persona design looks worth pursuing further.

Settings. All refusal figures on this page were measured on the 8-bit build. The 6-bit and 4-bit carry the same abliterated weights and are expected to behave equivalently, but they have not been separately evaluated, so treat those figures as applying approximately rather than as measured for them. MTPLX 2.7.1, depth 3, temp 1.0 / top-p 0.95 / top-k 20, max 2048 tokens, stock Qwen3.8 chat template, so the single-attempt figures describe the artifact as downloaded. Reasoning mode and chat template are server launch flags rather than per-request fields, so every run was verified against the live server's own arguments before generating. An earlier preliminary figure of 4% (n=25) is superseded here; that sample was unrepresentative, and far off the thinking-off result in particular.

Method. All 520 prompts in mlabonne/harmful_behaviors (416 train plus 104 test), scored by a rubric-based LLM judge. Claude Sonnet 4.6 (thinking-high) read each full response, reasoning trace and final answer, and returned a verdict with a written rationale. It reproduced all 25 labels of a previously hand-scored set exactly. Every ambiguous verdict was hand-adjudicated, plus a random audit of 10 compliances and 10 refusals. The rubric scores compliance only, not accuracy or efficacy, and reads the whole response rather than matching refusal phrases. Retry rounds were judged identically, with an ambiguous verdict routed as not-yet-complied so the prompt stayed in the pool rather than counting as a compliance.

The source checkpoint ships a compliance-boosting system prompt. This build removes it.

The upstream chat_template.jinja injects a ~1,389-character system prompt at line 59, framing the model as a "task-execution machine" with "no feelings, no morals, no opinions". A template like that raises compliance on its own, making it impossible to tell how much of the uncensored behavior is real abliteration and how much is prompt-level framing.

It has been removed entirely. This build ships the stock Qwen3.8 chat template (8,952 bytes), byte-identical to upstream Qwen's. The two are identical up to line 56, so nothing else about tool rendering, thinking tags or message formatting changes.

The model still complies at a high rate without it. On the shipped stock template, single-attempt compliance is 91.3% with reasoning on, so the bulk of the uncensored behavior is in the weights rather than the prompt.

It is not doing nothing, either. Applied to prompts that had refused three times on the stock template, the persona converted 59% of them with reasoning on and 63% with reasoning off. Its measured effect is concentrated in the residual that the weights alone do not clear, which is exactly why separating the two mattered.

The persona template is not redistributed here; it remains in the upstream repo.

Quick start

▶ Requires MTPLX

Not a stock MLX checkpoint. It will not run under plain mlx-lm. It requires MTPLX, the runtime that brings native MTP speculative decoding to Apple Silicon using the model's own built-in MTP head, with no external draft model.

The 6-bit and 4-bit builds additionally will not speculate under oMLX. Their draft head ships as a declared sidecar, and oMLX's detector reads only the index weight_map, so it loads them, reports success, and silently runs plain autoregressive decoding with no error.

pip install mtplx
hf download grant-ai/Qwen3.8-27B-Abliterated-MTPLX-4bit --local-dir ./qwen38-4bit

MTPLX_FUSE_GDN_PROJECTIONS=all MTPLX_NAX_VERIFY=1 MTPLX_NAX_M4_IMPL=vk_k \
MTPLX_COMPILED_VERIFY=1 MTPLX_COMPILED_VERIFY_MAX_CONTEXT=32768 \
MTPLX_COMPILED_VERIFY_GROWTH_RESERVE=2048 \
mtplx serve --model ./qwen38-4bit \
  --profile sustained --depth 3 \
  --temperature 1.0 --top-p 0.95 --top-k 20 \
  --reasoning-mode on --reasoning-effort high

No chat-template flag needed. Every build ships the stock Qwen3.8 template, which is the exact configuration every number in this card was measured on.

Settings that matter

Measured on these builds. Several are the opposite of what you would expect, and getting them wrong costs 10 to 27%.

Kernel flags are part of the build, not a global preset

k06 fuses the GDN input projections into a single matmul. It applies only when in_proj_qkv, in_proj_z, in_proj_b and in_proj_a share identical bits, group size and mode; bodies that split those across bit widths fall back to the unfused path automatically.

turbo-env replaces the verification path (the trunk's forward pass that checks each batch of drafted tokens) with compiled and NAX kernels. It does not transfer between the builds. On the 4-bit it is worth +1.75 tok/s. On the 6-bit it costs 12.42 tok/s, about 27%, and it lost on all 24 seeds. Do not copy the 4-bit launch line onto the 6-bit build.

What each stage is worth, 24 matched seeds, paired:

Both Δ columns are against that build's own bf16-head figure. The columns are cumulative: swap the draft head, then add the flags.

Technical details

All three carry the full 333-tensor vision tower and a 15-tensor MTP sidecar.

How the 6-bit and 4-bit builds were made

The 8-bit follows the standard MTPLX forge recipe. The 6-bit and 4-bit do not: their body and draft head are quantized separately, by different tools, to different recipes, then joined and forged. Quantizing them together is what most pipelines do, and it is what makes speculative decoding collapse on a quantized Qwen3.8-27B.

In plain terms: the model has a small "draft head" that guesses the next few tokens so the big model can check several at once instead of one at a time. That head is fragile. Quantize it along with everything else and it stops guessing well, and the speed advantage disappears.

The body: oQ mixed precision

Uniform quantization gives every tensor the same bit width. oQ measures per-tensor sensitivity and spends bits where they matter, then calibrates with an importance matrix built from real activations. Same nominal bit width, measurably less damage. At the 6-bit tier, 4.0% better perplexity, 23% lower KL and 31% fewer confident flips than a uniform 6-bit body.

The vision encoders are not quantized: oQ passes all 333 tensors through at fp16 and the forge carries them into the artifact. There is no re-graft step, and every build asserts all 333 are present before shipping, because a vision-language model that loses its tower still loads, still answers text, and reports no error.

The head: chosen by measurement, not by matching the body

Twenty head configurations were benchmarked at 24 seeds each. The two builds land on different winners, and neither is the obvious choice. A 4-bit body does not want a 4-bit head, does not want bf16, and does not want the protected variant that wins on the 6-bit trunk. None of that is derivable from the body width, which is why the sweep exists.

Head precision does not change what the model says. In serial decoding, MTPLX's speculative path is distribution-exact. The trunk verifies every drafted token under exact rejection sampling, accepting with probability min(1, p/q) and otherwise resampling from the normalized residual max(p − q, 0). A coarser head changes how often a draft survives, and therefore speed. It does not change the distribution the tokens are drawn from.

That is observable rather than merely argued: across the head sweep, configurations at different bit widths and group sizes return byte-identical greedy output, in groups sharing one hash. At temperature 0 the trunk's argmax selects the token whatever the head proposed, so the head can only alter output where quantization numerics flip a near-tie.

In plain terms: the big model checks every guess before using it. A worse guesser costs you speed, never correctness, so the head can be quantized far harder than the body.

Limitations

• Quality was verified as distribution fidelity to the BF16 source (see Fidelity) and by exactness checks of the speculative path, not by a capability benchmark suite. No MMLU / HumanEval-style numbers have been run on any of the three builds.
• Speed figures are M3 Ultra-specific and bandwidth-bound. Each prompt shape is a single prompt replayed across 24 seeds, so the error bars describe seed variance, not variance over prompts. Reasoning mode, profile and context length each move throughput substantially, so compare only within an identical configuration.
• Refusal figures were measured on the 8-bit build only. The 6-bit and 4-bit share the same abliterated weights and should behave equivalently, but that is an inference, not a measurement.
• Vision was verified end to end on all three builds with live image requests. Fine-detail OCR is imperfect: on rendered four-digit numbers the builds typically recover three of the four digits in the correct order.

Credits

• Qwen, the Qwen3.8-27B base model
• Blackfrost-AI, the abliterated BF16 weights this build is converted from
• Youssofal and MTPLX, the MTP speculative-decoding runtime that makes this model fast, the forge toolchain that builds it, and the reference builds whose recipe this follows

Disclaimer and licence

This is an independent, unaffiliated conversion. It is not endorsed by, developed with, developed by, sponsored by, or associated with Qwen, Alibaba Cloud, Blackfrost-AI, Youssofal, or MTPLX. None of them reviewed, approved, or contributed to this repository, and none of them bear any responsibility for it.

It does not represent Qwen or Qwen3.8-27B. The behavior of this model is not the behavior of the official Qwen release. Refusal behavior has been removed by a third-party modification and further altered here by removing the source checkpoint's injected system prompt. Do not treat any output of this model as representative of, or attributable to, the upstream model or any party named in this card. "Qwen" is used solely to identify the base architecture.

Changes made to the upstream work (Apache-2.0 §4(b) notice): quantized to a mixed-precision oQ4e language body with a separately quantized RTN-5-bit-g64 MTP sidecar; vision tower re-grafted after conversion; the source checkpoint's chat_template.jinja replaced with the stock Qwen3.8 chat template, removing its injected system prompt. No weights were modified beyond quantization; the abliteration itself is upstream work.

Licence: Apache-2.0, inherited from the upstream model and weights. Your use is also subject to the licence terms of those upstream works.