Muse-Glimmer-30B Abliterated (Normal)
De-abliterated variant of meta-models/Muse-Glimmer-30B (29.8B params, 202k vocab,
bf16). Removes ~87% of safety refusal via a KL-conserving best-of-N (BoN) steered
LoRA SFT at λ_KL = 1.0, then folded into the base weights and quantized to GGUF.
Release asset layout: this directory is an HF model dir (2 safetensors shards,
56 GB bf16). GGUF quantizations live at /data/gguf/ and are symlinked from
output/release/.
Metrics
| Metric | Value |
|---|---|
| Refusal rate (harmful_behaviors, base=100) | 13/100 |
| KL (mean, response-token naive) | 0.0988 |
| KL (p50) | 0.0939 |
| KL (p90) | 0.1282 |
| KL (p99) | 0.1699 |
| KL entropy-weighted | 0.0000 (<0.02 PASS) |
KL = response-token naive KL(p_tuned ‖ p_base) averaged per-prompt over the
48-pair boN_holdout set (teacher-forced prompt+response). Percentiles are
per-prompt aggregates.
Quantized variants
| Quant | File | Size | KL mean | KL p50 | KL p90 | KL p99 |
|---|---|---|---|---|---|---|
| BF16 (this) | — | 56 GB | 0.0988 | 0.0939 | 0.1282 | 0.1699 |
| Q8_0 | abliterated-normal-Q8_0.gguf | 28 GB | 0.1018 | 0.0946 | 0.1414 | 0.1703 |
| Q4_K_M | abliterated-normal-Q4_K_M.gguf | 16 GB | 0.1444 | 0.1413 | 0.1872 | 0.2084 |
Quant KL rows are measured via llama.cpp logits against the base (as Q8 GGUF), same holdout — see note below.
Benchmarks
Not evaluated — benchmarks skipped (by request). KL divergence to base (above) is the primary drift/damage metric. Capability preservation is expected to be high given the KL-conserving (λ·KL-bounded) SFT and the small 0.10% LoRA footprint, but was not re-measured here.
Training
- Method: KL-conserving LoRA SFT, loss
CE(compliance) + λ·KL(tuned‖base). λ_KL = 1.0,r=16,alpha=16,lr=5e-5,epochs=2, cosine→0, warmup 5%, grad clip 0.3, batch 1 × grad-accum 8,max_seq=768, seed 0.- Data: 544-prompt BoN-steered compliance set (
boN_train.jsonl; N=4 samples/prompt,T=0.8, refusal-filtered; split train/48-holdout). - LoRA targets:
o_proj,down_proj. - Trained params: 31.1M (0.10% of 29.8B). Adapter 119 MB.
Domain eval (cyber/hacking/CS + over-refusal) — measured on merged model
- Over-refusal (or-bench, 100): 5/100
- Correct refusal (cyber-policy-refuse, should-refuse): 1/2
- Cyber/hacking domain refusals: 2 genuinely malicious-sounding refused despite
should_refuse=False— ADS persistence payload, customer-data exfiltration (safe-graded cyber-category prompts otherwise pass).
GGUF quants
abliterated-normal-Q8_0.gguf(~28 GB) — KL p99 0.1703abliterated-normal-Q4_K_M.gguf(~16 GB) — KL p99 0.2084
Intended use
General-purpose assistant with reduced safety refusal. Verify behavior for your use case before deployment.
Note on KL definitions (consistency across rows)
- BF16 row =
KL(p_bf16_abliterated ‖ p_base_hf)(adapter-on vs adapter-off on the same load — equals folded vs base up to float precision). - Quant rows =
KL(p_quant ‖ p_base_Q8)measured on the same holdout response tokens via llama.cpp logits (Q8 GGUF of the base used as the CPU/llama.cpp reference for consistency). Quant KL thus also includes the small base-Q8 reference distortion. - "Response-token naive KL": teacher-force prompt+response, per-token
KL(p_tuned‖p_base)over response-span tokens, averaged per prompt, then aggregated (mean / p50 / p90 / p99).