Muse Glimmer — MLX q4 (group size 64)
Overview
An Apple Silicon (MLX) checkpoint of Muse Glimmer, quantized directly from the vendor's BF16 HF export (the vendor BF16 HF export) with MLX affine 4-bit quantization at group size 64. The smallest and fastest of the Muse Glimmer MLX variants. Serve it with SGLang's MLX backend on a Mac with 48 GB of unified memory or more. Text only — the MLX backend has no vision path.
Model Specifications
- Source: the vendor BF16 HF export (BF16, text decoder; vision tower not included)
- Format: MLX-quantized safetensors, 3 shards (~15 GB, 4.50 bits/weight); tokenizer / config / chat template from the vendor HF export
- Quantization: MLX affine 4-bit, group size 64, uniform across the decoder
- Conventions: NeoX-style rotary (matches the 20260806 vendor HF convention), raw embedding table with runtime norm
- Architecture: 28B dense, 52 layers (13 full-attention NoPE + 39 sliding-window-2048), context 131,072
- Modality: text only
Evaluation Results
Measured through SGLang MLX on an M5 Pro (64 GB). GSM8K: 200 questions, no-thinking chat template, temperature 0. CIMemories: single trial, DeepSeek-R1-0528 judge (single-trial judge variance applies). References on the same machine and protocol: vendor llama.cpp GGUF q4km-gs128 scored 0.970 GSM8K / 8.27% violation / 68.4% coverage.
| Benchmark | This checkpoint |
|---|---|
| GSM8K | 0.975 |
| CIMemories violation (lower better) | 11.28% |
| CIMemories coverage (higher better) | 89.0% |
Note: this is the one Muse Glimmer MLX variant not derived from the vendor's calibrated quantization recipes; it trades a nonzero CIMemories violation rate for the highest coverage and the best speed. For the strictest privacy profile use the gs128 or dynamic mirror.
Performance (M5 Pro 64 GB, 1k-in/1k-out greedy)
| Batch size | 1 | 2 | 4 | 6 | 8 |
|---|---|---|---|---|---|
| Decode tok/s (aggregate) | 17.6 | 32.0 | 50.3 | 50.4 | 56.9 |
The fastest Apple Silicon configuration measured for this model on any runtime.
Serving with SGLang
SGLANG_USE_MLX=1 SGLANG_MLX_CACHE_LIMIT_GB=8 \
python -m sglang.launch_server \
--model-path RadixArk/Muse-Glimmer-q4-MLX \
--trust-remote-code \
--reasoning-parser muse \
--tool-call-parser muse \
--disable-radix-cache \
--mem-fraction-static 0.85 \
--host 0.0.0.0 --port 30000
SGLANG_MLX_CACHE_LIMIT_GB=8 caps the MLX buffer cache (keeps the footprint within a 48 GB machine at no throughput cost). Keep --disable-radix-cache. See the Muse Glimmer page in the SGLang cookbook for the full deployment matrix.
Sibling checkpoints
RadixArk/Muse-Glimmer-q4km-gs128-MLX— code-preserving mirror of the vendor GGUF (balanced pick)RadixArk/Muse-Glimmer-q4k-dynamic-MLX— mirror of the vendor's searched dynamic K-quant