Qwen3.8-27B-heretic-MLX-bf16
Unquantized (bfloat16) MLX conversion of trohrbaugh/Qwen3.8-27B-heretic-ara, for Apple Silicon.
What this is
The base model is an abliterated ("heretic") derivative of Qwen/Qwen3.8-27B: its refusal behaviour has been surgically removed, so it will answer prompts a safety-tuned model would decline. This repo only changes the format and precision — it adds no alignment, and removes none either. Evaluate it before putting it in front of users, and apply your own filtering where your use case needs it.
Lineage: Qwen/Qwen3.8-27B → trohrbaugh/Qwen3.8-27B-heretic-ara → this MLX build.
Text-only
The base model is multimodal (image-text-to-text). This conversion is text-only.
mlx-lm converts the language tower only: the config carries no vision_config, and no
tensor belongs to the vision tower. Image and video input will not work.
No quantization
This is the unquantized MLX build: weights stay in bfloat16, exactly as the source
had them. It is the reference point for judging quality loss in the quantized builds,
and the slowest and largest of the set. For everyday use on Apple Silicon the 4-bit
build is 3x faster and needs a third of the memory.
The full set
All four builds were converted from the same source revision and measured the same way.
| Build | Size | Bits/weight | Generation | Peak memory |
|---|---|---|---|---|
| 4-bit | 15.1 GB | 4.501 | 37.9 tok/s | 15.5 GB |
| 6-bit | 21.9 GB | 6.501 | 27.9 tok/s | 22.2 GB |
| 8-bit | 28.6 GB | 8.501 | 22.2 tok/s | 28.9 GB |
| bf16 | 50.0 GB | 16 | 12.7 tok/s | 54.1 GB |
Measured on a Mac Studio M3 Ultra (512 GB unified memory), macOS 26.5.2, mlx-lm 0.31.3,
68-token prompt, 120 generated tokens. Single run, one machine, one prompt — an
order-of-magnitude guide, not a benchmark.
This repo is the bf16 build: 50.0 GB, 11 shards, 12.7 tok/s, 54.1 GB peak memory.
Usage
pip install mlx-lm
mlx_lm.generate --model donedynamics/Qwen3.8-27B-heretic-MLX-bf16 \\
--prompt "Introduce yourself briefly." --max-tokens 256
from mlx_lm import load, generate
model, tokenizer = load("donedynamics/Qwen3.8-27B-heretic-MLX-bf16")
messages = [{"role": "user", "content": "Introduce yourself briefly."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))
Reasoning mode
The chat template supports enable_thinking and reasoning_effort. Thinking is on by
default and consumes tokens before the answer begins, so a small max_tokens budget may
return reasoning only. Pass enable_thinking=False to apply_chat_template for direct
answers.
Conversion
mlx_lm.convert --hf-path trohrbaugh/Qwen3.8-27B-heretic-ara \\
--mlx-path Qwen3.8-27B-heretic-MLX-bf16
Converted with mlx-lm 0.31.3 from the bf16 weights at revision
a67ae100d933c0d17af3232bda35825979fc63ce. Before conversion the source was verified:
7 shards, 1199 tensors, all safetensors headers parsed, no missing files.
License and attribution
Apache-2.0, inherited through the lineage above. Credit for the model goes to the Qwen
team at Alibaba Group's Tongyi Lab, and for the abliteration to trohrbaugh. This repo
contributes only the MLX conversion.