Qwen3.6-14B-A3B-FableVibes-mlx-q4

MLX 4-bit quantization of tvall43/Qwen3.6-14B-A3B-FableVibes, for local inference on Apple Silicon.

Credit / original model

This repo is only a quantized MLX conversion. All credit for the model itself goes to the original author, tvall43. Please see and cite the original model card.

The base is a REAP-pruned Qwen3.6-35B-A3B reduced to ~14B total / ~3B active (90 experts, 8 active), recovered with a QLoRA distill of Claude Fable 5 reasoning traces. It uses the Qwen3.5 hybrid architecture (GatedDeltaNet linear attention + full attention + MoE) and emits <think>...</think> reasoning.

What this conversion did

• Fused the routed-MoE experts from per-expert tensors (experts.{i}.{gate,up,down}_proj) into mlx-lm's stacked experts.gate_up_proj / experts.down_proj format.
• Quantized to 4-bit, group size 64 with mlx-lm.
• ~7.2 GB; runs on a 16 GB Apple Silicon Mac.

Usage

uv run python -m mlx_lm generate \
  --model khanh2023/Qwen3.6-14B-A3B-FableVibes-mlx-q4 \
  --prompt "Solve: ..."

Notes

• MoE sparsity (~3B active/token) makes decode fast (~46 tok/s on an M4) despite 14B total params.
• 4-bit trades some exactness on strict reasoning; a higher-precision -mlx-q6 variant is also available.