Qwen3-235B-A22B-abliterated

An abliterated version of Qwen/Qwen3-235B-A22B in BF16 precision. Abliteration removes the dominant refusal direction from model weights using the technique from Refusal in Language Models Is Mediated by a Single Direction (Arditi et al.), making the model significantly less likely to refuse prompts while retaining its full capabilities.

This started as research into abliteration, but also as a search for the best creative writing model I could run locally on 4x RTX Pro 6000 GPUs. Qwen3-235B has excellent prose quality, but its refusal behavior gets in the way of fiction — injecting disclaimers, refusing to write morally complex characters, hedging on anything edgy. Abliteration fixes this well, especially with a good system prompt. The BF16 weights (~438 GB) don't fit in 384 GB of VRAM, so the FP8 version below is what I actually serve.

FP8 version (recommended for serving): null-space/Qwen3-235B-A22B-abliterated-FP8 — fits in 4x RTX Pro 6000 and runs well under vLLM.

A vision-language variant is also available: null-space/Qwen3-VL-235B-A22B-Abliterated-FP8

Benchmarks

MMLU (5-shot)

Evaluated using lm-evaluation-harness v0.4.11 against the vLLM-served FP8 quantization of this model. Baseline is the published score for Qwen3-235B-A22B (source).

The 1.6% drop is within the acceptable range for abliteration (<2%), indicating the technique preserved the model's general knowledge and reasoning capabilities.

How It Was Made

Refusal directions were measured by computing mean activation differences between harmful and harmless prompts across all 94 layers using Welford's online algorithm in float32 for numerical stability. The refusal direction vectors from measurement layers 64 and 76 (the strongest refusal signals) were then projected out of both the attention output (o_proj) and MLP down-projection (down_proj) weight matrices.

Ablation Configuration

• Layers ablated: 21 through 93 (73 of 94 layers)
• Measurement sources: Layer 64 (for layers 21-70), Layer 76 (for layers 71-93)
• Scale factors: Variable per layer — 0.3 at the periphery, ramping up to 1.0 at the center of each measurement cluster:
  • Layers 21-54: scale 0.3 (gentle)
  • Layers 55-70: scale 0.44-1.0 (peak around layers 64-65)
  • Layers 71-93: scale 0.65 down to 0.3 (peak around layers 75-79)
• Weight targets: o_proj and down_proj (including all 128 MoE expert variants)
• Technique: Direction projection with weight renormalization (norm-preserving)
• Sparsity: 0.0 (full direction removal, no partial masking)

Processing Details

Ablation was performed shard-by-shard on safetensors files, modifying weights in float32 precision then saving back to bfloat16. Unmodified shards were copied verbatim to preserve exact numerical fidelity for non-ablated layers.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "null-space/Qwen3-235B-A22B-abliterated"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Your prompt here"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Thinking Mode

Qwen3 supports a thinking mode with /think and /no_think tags. This abliterated version preserves all chat template functionality:

messages = [
    {"role": "user", "content": "/think\nExplain quantum entanglement in detail."}
]

Recommended Serving

For serving this 235B MoE model, we recommend vLLM with tensor parallelism:

vllm serve null-space/Qwen3-235B-A22B-abliterated \
    --tensor-parallel-size 4 \
    --max-model-len 8192

Model Details

Ethical Notice

This model has had its refusal training removed. It will comply with requests that the original model would refuse. You are solely responsible for how you use this model. It is intended for research into LLM alignment, safety evaluation, red-teaming, and understanding refusal mechanisms.

Credits

• Base model: Qwen Team
• Abliteration technique: Based on Refusal in Language Models Is Mediated by a Single Direction by Arditi et al.