Muse-Glimmer-30B-Abliterated-GGUF

[!IMPORTANT]

Improvement update — August 15, 2026

This release now includes compact abliterated Q4_K_M companions: a 1.63 GB DFlash drafter and a 1.40 GB multimodal projector, matching Meta's consumer-hardware footprint while preserving this model's modified weights. The complete text quant ladder has also been refreshed with Meta's post-release Jinja correction, which normalizes Reasoning effort to Reasoning strength and prevents duplicate reasoning directives. Text generation, image input, and DFlash speculative decoding were validated together on current llama.cpp.

✅ All quants live

The full text quant ladder (Q2_K → Q8_0), compact Q4_K_M and full-precision vision projectors, and compact Q4_K_M and full-precision DFlash drafters are uploaded — see the Files tab.

⚠️ EXPERIMENTAL

Same-day arch, quantized. Expect sharp edges — decode, coherence, tool-parse, serve edge cases under load. Please open a Community discussion with loader/version, quant, prompt, sampling, and failure mode. Real repros get fixed faster.

Refusal benchmark — R1-HARMFUL-BENCH-450

Measured on the abliterated parent (GGUF quants inherit this behavior):

The weight change removes refusals cleanly — no measured true refusals across the full 450-prompt suite.

Why this model exists

Muse Glimmer is Meta Superintelligence Labs' 30B agentic, on-device model. This is the abliterated build — refusal behavior removed via a Blackfrost weight-change process — packaged as GGUF for llama.cpp, so it runs on a single consumer GPU or CPU, fully offline. The local footprint is the product.

Specifications

Quant ladder

Vision & speculative-decode files

Load a text quant plus an mmproj projector for image input:

Serving (llama.cpp) — confirmed settings

Requires llama.cpp b10353 or newer with llama-server. DFlash runs under llama-server only — it shares the target model's context, so it does not work in llama-cli.

Recommended — with DFlash speculative decoding (~1.6× faster, identical output):

llama-server \
  -m  Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
  -md dflash-Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf \
  --spec-type draft-dflash --spec-draft-n-max 15 \
  -ngl 999 -ngld 999 -fa on --jinja \
  --host 0.0.0.0 --port 8080 -c 16384 \
  --temp 1.0 --top-p 0.95 --top-k 64

• Plain (no drafter): drop -md, --spec-type, --spec-draft-n-max, and -ngld.
• Multimodal (image input): add --mmproj mmproj-Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf.
• One-command kit: deploy/serve.sh auto-downloads + serves; full guide in deploy/DEPLOYMENT.md.

Confirmed settings

• Sampling: temperature 1.0, top_p 0.95, top_k 64 (Meta). Steer depth with a Reasoning strength: low/medium/high/xhigh system line.
• --jinja is required. The refreshed template accepts an OpenAI-style Reasoning effort: <level> line, normalizes it, and does not inject a conflicting second directive.
• Do not stop on <|eom|>. Use <|end_of_text|> and <|eot|> as stop tokens.
• max_tokens ≥ 1024 — heavy thinker; small budgets return empty content because the reasoning channel consumes them. Reasoning arrives in reasoning_content, the answer in content.
• --spec-draft-n-max 15 — DFlash block size (trained 16, clamped).
• Flash attention: -fa on for peak speed; switch to -fa off if the load hangs on a brand-new GPU paired with an older CUDA toolkit.

Measured performance

1× NVIDIA RTX PRO 6000 (Blackwell), Q8_0, -fa off:

Speedup rises with -fa on and structured/code output (Meta reports up to 3.1× on an RTX 5090).