Qwen3.8-27B-heretic-VL-MLX-bf16
Unquantized (bfloat16) MLX conversion of trohrbaugh/Qwen3.8-27B-heretic-ara, with the vision tower intact, for Apple Silicon.
Vision works — and was tested
This build keeps the vision encoder: vision_config is present and 333 of the 2180
tensors belong to vision_tower. It reads images.
Verified rather than assumed. A generated test image (red square left, blue circle
centre, green triangle right, the caption MAC512 VISION TEST, and the code
7391-ZQ) was fed to all four builds. Every one named the three shapes with the right
colours and positions and read both strings back exactly.
Requires mlx-vlm, not mlx-lm. mlx-lm converts the language tower only and
would silently drop vision. Text-only siblings live at
Qwen3.8-27B-heretic-MLX-*
— smaller and slightly leaner if you never send images.
What this is
The base model is an abliterated ("heretic") derivative of Qwen/Qwen3.8-27B: its refusal behaviour has been surgically removed, so it will answer prompts a safety-tuned model would decline. This repo only changes the format and precision — it adds no alignment, and removes none either. Evaluate it before putting it in front of users, and apply your own filtering where your use case needs it.
Lineage: Qwen/Qwen3.8-27B → trohrbaugh/Qwen3.8-27B-heretic-ara → this MLX build.
The full set
| Build | Size | Bits/weight | Generation | Peak memory |
|---|---|---|---|---|
| 4-bit | 15.0 GB | 4.695 | 38.9 tok/s | 19.2 GB |
| 6-bit | 21.3 GB | 6.661 | 29.2 tok/s | 27.0 GB |
| 8-bit | 27.5 GB | 8.627 | 23.1 tok/s | 34.7 GB |
| bf16 | 51.0 GB | 16 | 13.2 tok/s | 55.8 GB |
Measured on a Mac Studio M3 Ultra (512 GB unified memory), macOS 26.5.2, mlx-vlm 0.6.13,
470-token multimodal prompt (image + text). Single run, one machine, one prompt — an
order-of-magnitude guide, not a benchmark. Prompt processing ran at 303-331 tok/s across
all four builds; the spread between builds shows up in generation, not prefill.
This repo is the bf16 build: 51.0 GB, 11 shards, 13.2 tok/s, 55.8 GB peak memory.
Peak memory runs roughly 4-6 GB above the text-only sibling — that is the vision tower plus image-patch activations.
Usage
pip install mlx-vlm
mlx_vlm.generate --model donedynamics/Qwen3.8-27B-heretic-VL-MLX-bf16 \\
--image photo.png --prompt "What does this image show?" --max-tokens 256
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("donedynamics/Qwen3.8-27B-heretic-VL-MLX-bf16")
prompt = apply_chat_template(processor, model.config, "What does this image show?", num_images=1)
print(generate(model, processor, prompt, ["photo.png"], max_tokens=256))
Reasoning mode
The chat template supports enable_thinking and reasoning_effort. Thinking is on by
default and consumes tokens before the answer begins, so a small max_tokens budget may
return reasoning only.
Conversion
mlx_vlm.convert --hf-path trohrbaugh/Qwen3.8-27B-heretic-ara \\
--mlx-path Qwen3.8-27B-heretic-VL-MLX-bf16
Converted with mlx-vlm 0.6.13 from the bf16 weights at revision
a67ae100d933c0d17af3232bda35825979fc63ce. Before conversion the source was verified:
7 shards, 1199 tensors, all safetensors headers parsed, no missing files.
License and attribution
Apache-2.0, inherited through the lineage above. Credit for the model goes to the Qwen
team at Alibaba Group's Tongyi Lab, and for the abliteration to trohrbaugh. This repo
contributes only the MLX conversion.