Muse-Glimmer-30B NVFP4 GGUF
Base Model
Muse-Glimmer-30B by Meta — a 27.8B-parameter dense causal transformer with integrated vision encoder (ViT-G/14, ~1.8B params). Apache 2.0 licensed. Supports images, screenshots, charts, documents, and 131K+ context. No MTP heads — uses DFlash speculative decoding instead.
Vision Support
This model includes a full vision encoder (mmproj) embedded in the GGUF. Use llama-mtmd-cli or llama-server --mmproj for vision capabilities. The vision encoder uses dynamic-size input with patch size 14 and temporal patch 2.
Quantization
- Format: NVIDIA FP4 (NVFP4) — 4-bit E2M1 values with E4M3 scaling per 16 values + FP32 tensor scale
- BPW: 4.69 bits per weight
- Size: 15.6 GB
- Target hardware: NVIDIA Blackwell GPUs (RTX 50 series) — requires CUDA kernels at inference
Tested Settings (llama.cpp build 1647+, RTX 5060 Ti 16GB)
llama-cli -m muse-glimmer-30b-nvfp4.gguf \
-ngl 99 \
--ctx-size 131072 \
--flash-attn on \
--cache-type-k q8_0 --cache-type-v q8_0 \
--reasoning off \
-t 8 --temp 0.7 --top-k 20 --top-p 0.95
Performance Results
| Metric | Result |
|---|---|
| Prompt processing | 80.2 t/s |
| Token generation | 8.7 t/s |
| Context | 128K |
| KV Cache | Q8_0 |
| Test prompt | "Explain to me in detail what is superpositioning and Quantum computing" |
| Output quality | Coherent, detailed, accurate — covered Dirac notation, Bloch sphere, interference, quantum parallelism |
Usage
llama.cpp CLI (text generation)
llama-cli -m muse-glimmer-30b-nvfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off -p "Your prompt here"
llama.cpp Server (API)
llama-server -m muse-glimmer-30b-nvfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off --host 0.0.0.0 --port 8080
Vision (requires llama-mtmd-cli)
llama-mtmd-cli -m muse-glimmer-30b-nvfp4.gguf -ngl 99 --flash-attn on --reasoning off --image /path/to/image.png -p "Describe this image"
Requirements
- llama.cpp build 1647+ with CUDA support (Muse-Glimmer architecture support merged Aug 2026)
- NVIDIA GPU with 16GB+ VRAM (tested on RTX 5060 Ti 16GB)
- Full GPU offload works — model fits entirely in VRAM
--reasoning offrecommended to suppress thinking token leakage
Notes
- Dense model (all 27.8B params active per token) — no
--cpu-moeneeded - Sliding window attention (2048 for 75% of layers, full for 25%) keeps KV cache small at 128K
- For faster generation (~3x), see the MXFP4 variant
- Reasoning cannot be fully disabled via config — use
--reasoning offat inference
License
Apache 2.0 (inherited from base model)