Qwen3.8-27B UD-Q2_K_XL — MLX imatrix-direct (native MLX, dynamic quant)
Native-MLX port of unsloth's UD-Q2_K_XL dynamic quantization for
Qwen/Qwen3.8-27B. Runs the quantizer search natively in MLX's affine format
(group size 64) and writes final MLX weights directly — no GGUF round-trip,
no mlx_lm.convert re-quantization.
Perplexity (wikitext-2-raw first 32k tokens, 512-token windows):
| Build | PPL ↓ |
|---|---|
| unsloth UD-Q2_K_XL GGUF in llama.cpp | 7.180 |
| this model | 9.133 |
Note: at 2 bits the gap to the GGUF widens (+1.95) because unsloth's 2-bit quants use non-linear IQ codebooks that MLX's affine kernels cannot represent (details).
Method
- Bit map from unsloth's
Qwen3.8-27B-UD-Q2_K_XL.gguf(dynamic per-tensor bit allocation). - Importance-weighted affine search per group using unsloth's published imatrix (per-column activation energies).
- Final MLX weights emitted directly (packed uint32 codes + bf16 scales/biases) — the emitted weights are exactly what the search chose.
Limitations
- Affine vs IQ-codebook gap. MLX quantized matmul supports only uniform affine grids; unsloth's ≤3-bit GGUFs use non-linear IQ codebooks. The gap vs GGUF is inherent to MLX's current kernels, not the weights.
- Text-only. The base checkpoint's visual tower is not included.
- Eval scope: wikitext-2, 32k tokens only; downstream tasks uncharacterized.
- Not for further training.
Calibration data
Importance weighting: unsloth's published imatrix_unsloth.gguf.
Credits
- unsloth — dynamic bit maps + imatrix
- mlx-lm — MLX runtime
- Pipeline: jclyons52/ud2mlx
Built 2026-08-28. Base model license inherited from Qwen/Qwen3.8-27B.