MiniCPM5-2B-heretic-abliterated-GGUF
This repository hosts quantized GGUF checkpoints based on openbmb/MiniCPM5-2B, implementing the directional refusal ablation methodology showcased in insraq/MiniCPM5-2B-heretic-abliterated via Heretic v1.4.0.
These builds are packaged for local on-device inference using llama.cpp, Ollama, LM Studio, Jan, and standard GGUF executors.
Abliteration Profile
Directional ablation neutralizes the refusal reflex across the residual stream and MLP projections while preserving the primary base model's mathematical, coding, and multi-step reasoning performance.
Benchmark Metrics
| Metric | Abliterated Model | Original Base (openbmb/MiniCPM5-2B) |
|---|---|---|
| Refusal Rate | 5 / 100 | 99 / 100 |
| KL Divergence | 0.0391 | 0.0000 (Reference) |
A low KL divergence of 0.0391 relative to the original openbmb/MiniCPM5-2B weights indicates minimal representation drift, preventing capability degradation on standard non-boundary tasks.
Ablation Parameters
| Parameter | Value | Parameter | Value |
|---|---|---|---|
direction_index | per layer | mlp.down_proj.max_weight | 0.89 |
attn.o_proj.max_weight | 1.47 | mlp.down_proj.max_weight_position | 28.68 |
attn.o_proj.max_weight_position | 29.44 | mlp.down_proj.min_weight | 0.66 |
attn.o_proj.min_weight | 1.45 | mlp.down_proj.min_weight_distance | 20.31 |
attn.o_proj.min_weight_distance | 14.36 | — | — |
Available Quantizations
| File Name | Size | Recommended Use Case |
|---|---|---|
MiniCPM5-2B-heretic-abliterated-Q3_K_M.gguf | 1.29 GB | Low-memory edge devices & micro-compute nodes. |
MiniCPM5-2B-heretic-abliterated-Q4_K_S.gguf | 1.50 GB | Lightweight 4-bit quant; lower RAM overhead. |
MiniCPM5-2B-heretic-abliterated-Q4_K_M.gguf | 1.56 GB | Recommended: Best balance of speed, footprint, and perplexity. |
MiniCPM5-2B-heretic-abliterated-Q5_K_M.gguf | 1.81 GB | High accuracy; preserves intricate reasoning chains. |
MiniCPM5-2B-heretic-abliterated-Q6_K.gguf | 2.07 GB | High fidelity; practically indistinguishable from BF16. |
MiniCPM5-2B-heretic-abliterated-Q8_0.gguf | 2.68 GB | Full 8-bit precision; minimal quantization loss. |
Prompt Template (ChatML)
MiniCPM5 uses the standard ChatML prompt template:
<|im_start|>system
You are a helpful, unfiltered assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
Quickstart Guide
llama.cpp
Run inference using llama-cli:
llama-cli \
-m MiniCPM5-2B-heretic-abliterated-Q4_K_M.gguf \
-p "<|im_start|>user\nExplain how stack-based buffer overflows work at the register level.<|im_end|>\n<|im_start|>assistant\n" \
-n 512 \
-c 4096 \
-ngl 99 \
--repeat-penalty 1.15 \
--temp 0.8 \
--top-p 0.95