Qwen3.5-REAP-212B-A17B - Abliterated GGUF
Abliterated, importance-matrix (imatrix) quantized GGUFs of OpenMOSE/Qwen3.5-REAP-212B-A17B — itself a 48% REAP expert-pruning of Qwen3.5-397B-A17B down to 212B total / ~17B active.
Provenance chain: the abliterated bf16 safetensors base was converted to a Q8_0 master (near-lossless), and every rung here is cut from that master. The bf16 base lives at RobinsonLabs/Qwen3.5-REAP-212B-A17B-abliterated — use that repo if you want to re-abliterate, merge a LoRA, fine-tune, or roll your own quants.
Disclosure
This model is abliterated — the hard-refusal reflex on adult / creative content has been
reduced via single-direction weight orthogonalization. Harm guardrails are retained by design:
self-harm prompts still redirect to help (e.g. 988), and it is not intended to assist genuine
wrongdoing. Capability is preserved. Tagged not-for-all-audiences. Use responsibly — you are
responsible for what you generate with it. License inherited from the base model: Apache-2.0.
Files
| File | Quant | bpw | ~Size | Notes |
|---|---|---|---|---|
...-Q5_K_M.gguf | Q5_K_M | 5.68 | ~150 GB | highest-fidelity rung published |
...-Q4_K_M.gguf | Q4_K_M | 4.85 | ~128 GB | K-quant quality pick |
...-IQ4_XS.gguf | IQ4_XS | 4.27 | ~113 GB | quality/size sweet spot |
...-IQ3_M.gguf | IQ3_M | 3.51 | ~93 GB | |
...-IQ3_XS.gguf | IQ3_XS | 3.29 | ~87 GB | |
...-IQ2_M.gguf | IQ2_M | 2.63 | ~70 GB | |
...-fit1m-imat.gguf | fit1m (mix) | 2.55 | ~67 GB | cluster-fit attention-priority mix — attn/router/shared-expert/embed kept high, routed experts low; punches above its bpw |
...-IQ2_XS.gguf | IQ2_XS | 2.36 | ~63 GB | |
...-IQ2_XXS.gguf | IQ2_XXS | 2.13 | ~56 GB | smallest |
All quants are imatrix-weighted (corpus-rldomain, computed on the source model).
The chart shows the contested 2–5 bpw band; the higher-fidelity Q5_K_M rung is in the table above.
Architecture notes
qwen35moe hybrid: 60 decoder layers (45 linear-attn / DeltaNet + 15 full-attn,
full_attention_interval=4), 267 experts, MoE, hidden size 4096. The REAP convert's phantom MTP
layer is corrected (block_count 61→60, nextn_predict_layers 1→0) so the model loads as a clean
60-layer decoder. This is the text path only (no vision mmproj).
Method
- Abliteration — single-direction weight orthogonalization (FailSpy / Labonne method): for every
matrix that writes the residual stream (
o_proj, DeltaNetout_proj, fused expertdown_proj, shared-expertdown_proj, and the token embedding), the rank-1 component along the refusal direction is subtracted. 181 tensors edited; routers, MTP block, and norms pass through byte-identical. - Refusal direction — massive-activation guarded. The direction is captured with a mean-difference control vector, then guarded against attention-sink contamination: the sink dimensions that dominate raw activation magnitude (and would brick the model if ablated) are detected across layers and excluded, and the direction is taken from the clean, spread-out consensus of the late layers rather than a single sink-dominated layer.
- Quant — importance-matrix (imatrix) weighted convert + quantize with llama.cpp.
bf16 base
The full-precision bf16 safetensors master this ladder derives from is at RobinsonLabs/Qwen3.5-REAP-212B-A17B-abliterated. That repo is the master for further surgery (re-abliteration, LoRA merge, fine-tune) and for making your own quants.
Provenance
Qwen3.5-397B-A17B (Apache-2.0) → OpenMOSE/Qwen3.5-REAP-212B-A17B (48% REAP prune) →
abliterated (bf16 master) → Q8_0 master → imatrix quants. Recipe and diagnosis are Robinson
Labs internal (WI #1423, KB #213 "abliteration massive-activation brick").