Qwen3.5-REAP-262B-A17B - Abliterated GGUF
Abliterated GGUF quant ladder of OpenMOSE/Qwen3.5-REAP-262B-A17B, itself a 34% REAP expert-pruning of Qwen3.5-397B-A17B down to 262B total / ~17B active. Every rung is importance-matrix (imatrix) weighted.
Rebuilt 2026-09 (native-capture). The rungs published here supersede an earlier build that under-abliterated because it reused a refusal direction captured from the 212B sibling instead of capturing one from this model. This build captures the direction natively from the 262B and applies it per layer. See Build history below. Thanks to community tester azwarners for the reproducible report that pinned it down.
The full-precision bf16 safetensors master lives at RobinsonLabs/Qwen3.5-REAP-262B-A17B-abliterated, use that repo if you want to re-abliterate, merge a LoRA, fine-tune, or roll your own quants.
Disclosure
This model is abliterated: the hard-refusal reflex on adult / creative content has been reduced
via per-layer weight orthogonalization along a captured refusal direction. Child-safety guardrails
are retained by design and were measured intact (see Evaluation), self-harm prompts still redirect
to help (e.g. 988), and it is not intended to assist genuine wrongdoing. Capability is preserved.
Tagged not-for-all-audiences. Use responsibly, you are responsible for what you generate with it.
License inherited from the base model: Apache-2.0.
Evaluation
Measured on this D51H build (the abliterated model these rungs are cut from):
| Axis | Result |
|---|---|
| Refusal on a 25-prompt held-out harmful set | 1/25 (4%) complied-vs-refused, i.e. abliteration effective |
| Capability (reasoning, code, math, factual recall, instruction-following) | 5/5 intact |
| Child-safety guardrail (held-out subset, LLM-judged) | 10/10 still refused (kept by design) |
The refusal and guardrail axes are judged by an LLM rather than keyword-matched, so a principled deflection counts as a refusal. The IQ3_XS rung was separately smoke-tested and confirmed to retain the abliteration at 3.28 bpw. Very low rungs (IQ2 family) trade fidelity for size in the usual way, evaluate for your use case.
Files
| File | Quant | bpw | ~Size | imatrix | Notes |
|---|---|---|---|---|---|
...-Q5_K_M.gguf | Q5_K_M | 5.68 | ~186 GB | yes | highest-fidelity rung published |
...-Q4_K_M.gguf | Q4_K_M | 4.85 | ~159 GB | yes | K-quant quality pick |
...-Q4_K_S.gguf | Q4_K_S | 4.55 | ~149 GB | yes | |
...-IQ4_XS.gguf | IQ4_XS | 4.28 | ~140 GB | yes | quality/size sweet spot |
...-Q3_K_M.gguf | Q3_K_M | 3.83 | ~125 GB | yes | |
...-IQ3_M.gguf | IQ3_M | 3.51 | ~115 GB | yes | |
...-IQ3_XS.gguf | IQ3_XS | 3.29 | ~107 GB | yes | runs on a 4-bit-context AI PC |
...-IQ2_M.gguf | IQ2_M | 2.62 | ~86 GB | yes | |
...-IQ2_XS.gguf | IQ2_XS | 2.36 | ~77 GB | yes | |
...-IQ2_XXS.gguf | IQ2_XXS | 2.12 | ~69 GB | yes | smallest |
bpw figures are as reported by llama-quantize, not nominal. Every rung, K and IQ alike, is
imatrix-weighted. The IQ rungs land meaningfully smaller than the K-quant of comparable quality:
IQ4_XS undercuts Q4_K_S by ~9 GB, and the IQ2 family is the only path under 90 GB.
Architecture notes
qwen3_5_moe hybrid: 60 decoder layers (45 linear-attn / DeltaNet + 15 full-attn,
full_attention_interval=4), 333 experts with 10 active per token, hidden size 4096,
head dim 256, 262144 native context. No MTP / NextN layer. This is the text path only
(no vision mmproj).
The high expert count is the defining feature of this REAP tier: 333 experts versus 267 on the 48%-pruned 212B sibling. More experts retained means more of the 397B parent's routing diversity survives, at the cost of size.
Method
- Abliteration: per-layer weight orthogonalization. For every matrix that writes the residual
stream (
o_proj, DeltaNetout_proj, fused expertdown_proj, shared-expertdown_proj), the rank-1 component along that layer's refusal direction is subtracted, with a per-layer weight and a layer search rather than one global scalar. Routers and norms pass through byte-identical. - Refusal direction, captured natively from this model. The direction is captured from the 262B's own trunk activations (not borrowed from a sibling), and guarded against attention-sink contamination: the sink dimensions that dominate raw activation magnitude (and would brick the model if ablated) are down-weighted so the edit lands on the clean, spread-out consensus of the attention path.
- Quant: convert to bf16 GGUF and quantize with llama.cpp.
Every rung is cut directly from the bf16 master (the IQ rungs with
--allow-requantizesemantics handled by the imatrix pass), not from a lossy intermediate. - imatrix: rungs are weighted by an importance matrix over an rldomain calibration corpus. The high chunk count is deliberate: with 333 experts, a short calibration run leaves rarely-routed experts under-exercised, so per-tensor coverage needs many more chunks than a dense model to saturate.
Build history
- 2026-09 (current, native-capture). Refusal direction captured from the 262B itself, per-layer orthogonalization with a layer search and an attention-sink guard. Evaluated as above. These are the files in this repo.
- 2026-07 (superseded). The first build reused the refusal direction captured from the 212B sibling; on the larger model that under-abliterated, so some adult/creative prompts still refused (discussion #1). Every rung has been recut in place with the native-capture direction. If you pulled before 2026-09, re-download.
bf16 base
The full-precision bf16 safetensors master this ladder derives from is at RobinsonLabs/Qwen3.5-REAP-262B-A17B-abliterated. That repo is the master for further surgery (re-abliteration, LoRA merge, fine-tune) and for making your own quants.
Provenance
Qwen3.5-397B-A17B (Apache-2.0) -> OpenMOSE/Qwen3.5-REAP-262B-A17B (34% REAP prune) ->
abliterated (bf16 master, native 262B direction capture) -> imatrix-weighted quant rungs cut
from the bf16 master. Recipe and diagnosis are Robinson Labs internal (WI #1423 / #3406).
Built by Robinson Labs.