Ornith-1.5-35B-A3B-abliterated
Abliterated (refusal-direction removed) build of
ornith-ai/Ornith-1.5-35B-A3B, BF16, 65 GB.
An NVFP4 quantization of this model (20 GB, runs on 2×16 GB GPUs, with benchmarks) is at
pottokao/Ornith-1.5-35B-A3B-abliterated-NVFP4-DFlash.
⚠️ Text-only. This is a language-model-only export: it contains no vision tower and no MTP head, unlike the original release.
model_typeisqwen3_5_moe_text, architectureQwen3_5MoeForCausalLM.
⚠️ Uncensored. Safety refusal behaviour has been deliberately removed. You are responsible for how you use it.
Method
Classic refusal-direction ablation (orthogonalization), single direction:
| Step | Detail |
|---|---|
| Probe layer | 24 — int(num_layers × 0.6), 40 layers total |
| Samples | 64 harmful + 64 harmless prompts (random.seed(0)), last-token hidden state |
| Direction | d = normalize(mean(harmful) − mean(harmless)) |
| Ablation | For every .o_proj and .down_proj: W ← W − outer(d, dᵀW) |
The refusal direction is projected out of the output space of the attention- and MLP-output
projections, leaving all other weights untouched. Tooling derived from
remove-refusals-with-transformers.
Architecture
Hybrid Mamba + MoE, 40 layers: 30 linear-attention (Mamba-style) layers and 10 full-attention layers (indices 3, 7, …, 39), 256 experts per layer with a shared expert, ~3 B active parameters. Context length 262,144.