Muse Glimmer 30B - Heretic Abliterated (BF16)
v2 Release - Significantly improved abliteration using 500 Optuna trials with Heretic.
Results
| Version | Refusals | Compliance | KL Divergence | Trials |
|---|---|---|---|---|
| v2 (current) | 6.5% | 93.5% | 0.076 | 500 |
| v1 | 29% | 71% | 0.027 | 50 |
The v2 release achieves an 88% refusal reduction over v1 while maintaining strong model quality (KL=0.076).
Methodology
This model was abliterated using Heretic, a state-of-the-art refusal vector removal tool that uses LoRA adapters and Optuna hyperparameter optimization.
Abliteration Pipeline
-
Refusal Direction Computation: Refusal directions were computed across all 52 transformer layers by comparing residual stream activations between harmful and harmless prompts from the
mlabonne/harmful_behaviorsandmlabonne/harmless_alpacadatasets. -
Optuna Optimization: 500 trials were run optimizing for minimum refusal rate while preserving model quality (measured via KL divergence). Each trial configures weight parameters for the
attn.o_projandmlp.down_projcomponents across layers. -
LoRA Abliteration: The best trial parameters are applied as LoRA adapters to the target weight matrices, projecting out the refusal direction from the model's representations.
-
Weight Merging: LoRA adapters are merged back into the base weights, producing a clean BF16 model with no adapter overhead.
Best Trial (Trial 445 of 500)
- Refusal rate: 6.5% (93.5% compliance)
- KL divergence: 0.076
- direction_index: 40.73
Usage
from transformers import AutoModelForImageTextToText, AutoTokenizer
model = AutoModelForImageTextToText.from_pretrained(
"mlasli/Muse-Glimmer-30B-Heretic-Abliterated-BF16",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("mlasli/Muse-Glimmer-30B-Heretic-Abliterated-BF16")
Hardware Requirements
- VRAM: ~55 GB (BF16)
- Recommended: 1x A100 80GB or 2x A6000 48GB
GGUF Versions
Quantized GGUF versions of this model are available:
- Q4_K_M: mlasli/Muse-Glimmer-30B-Heretic-Abliterated-Q4_K_M-GGUF (~16 GB)
- Q6_K: mlasli/Muse-Glimmer-30B-Heretic-Abliterated-Q6_K-GGUF (~22 GB)
- Q8_0: mlasli/Muse-Glimmer-30B-Heretic-Abliterated-Q8_0-GGUF (~28 GB)
License
Apache 2.0 (same as base model Meta Muse Glimmer 30B)
Citation
- Base model: Meta Muse Glimmer 30B
- Abliteration tool: Heretic