Qwen3.8-27B-ABLITERATED — 3-bit MLX TextOnly
This is a 3-bit MLX TextOnly quantization of Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16, optimized for inference on Apple Silicon.
Quantization
- Format: MLX
- Quantization: 3-bit affine
- Group size: 64
- Vision tower: removed
- Runtime:
mlx-lm - Base model:
Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
The language-model weights were converted to MLX and quantized to 3-bit. The vision components of the original multimodal model are not included, making this release TextOnly and reducing memory requirements.
Usage
Install MLX LM:
pip install -U mlx-lm
Run an interactive chat:
mlx_lm.chat \
--model EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly
Or generate directly:
mlx_lm.generate \
--model EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly \
--prompt "Hello! Explain quantization briefly." \
--max-tokens 512
To disable thinking when supported by the model's chat template:
mlx_lm.generate \
--model EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly \
--prompt "Hello!" \
--max-tokens 512 \
--chat-template-config '{"enable_thinking": false}'
Credits
Original model and fine-tune:
Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
This repository only provides an MLX quantization/conversion of the original model. All credit for the underlying model and abliteration work belongs to the original authors.