Muse-Glimmer-30B MXFP4 GGUF
Base Model
Muse-Glimmer-30B by Meta — a 27.8B-parameter dense causal transformer with integrated vision encoder (ViT-G/14, ~1.8B params). Apache 2.0 licensed. Supports images, screenshots, charts, documents, and 131K+ context. No MTP heads — uses DFlash speculative decoding instead.
Vision Support
This model includes a full vision encoder (mmproj) embedded in the GGUF. Use llama-mtmd-cli or llama-server --mmproj for vision capabilities. The vision encoder uses dynamic-size input with patch size 14 and temporal patch 2.
Quantization
- Format: MXFP4 — OCP microscaled FP4 (E2M1 values with E8M0 power-of-two scaling per 32 values)
- BPW: 4.46 bits per weight
- Size: 14.8 GB
- Portable: MXFP4 works on any hardware with FP4 support (not Blackwell-only like NVFP4)
Tested Settings (llama.cpp build 1647+, RTX 5060 Ti 16GB)
llama-cli -m muse-glimmer-30b-mxfp4.gguf \
-ngl 99 \
--ctx-size 131072 \
--flash-attn on \
--cache-type-k q8_0 --cache-type-v q8_0 \
--reasoning off \
-t 8 --temp 0.7 --top-k 20 --top-p 0.95
Performance Results
| Metric | Result |
|---|---|
| Prompt processing | 246.3 t/s |
| Token generation | 25.3 t/s |
| Context | 128K |
| KV Cache | Q8_0 |
| Test prompt | "Explain to me in detail what is superpositioning and Quantum computing" |
| Output quality | Coherent, detailed, accurate — covered superposition, Dirac notation, entanglement, interference, quantum algorithms |
Usage
llama.cpp CLI (text generation)
llama-cli -m muse-glimmer-30b-mxfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off -p "Your prompt here"
llama.cpp Server (API)
llama-server -m muse-glimmer-30b-mxfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off --host 0.0.0.0 --port 8080
Vision (requires llama-mtmd-cli)
llama-mtmd-cli -m muse-glimmer-30b-mxfp4.gguf -ngl 99 --flash-attn on --reasoning off --image /path/to/image.png -p "Describe this image"
Requirements
- llama.cpp build 1647+ with CUDA support (Muse-Glimmer architecture support merged Aug 2026)
- NVIDIA GPU with 16GB+ VRAM (tested on RTX 5060 Ti 16GB)
- Full GPU offload works — model fits entirely in VRAM
--reasoning offrecommended to suppress thinking token leakage
Notes
- Dense model (all 27.8B params active per token) — no
--cpu-moeneeded - ~3x faster generation than NVFP4 variant (25.3 vs 8.7 t/s) — MXFP4 decode kernels are more efficient
- Sliding window attention (2048 for 75% of layers, full for 25%) keeps KV cache small at 128K
- For Blackwell-optimized variant, see the NVFP4 variant
- Reasoning cannot be fully disabled via config — use
--reasoning offat inference
License
Apache 2.0 (inherited from base model)