Qwen3.8-27B-RANA-abliterated

A refusal-ablated version of Qwen/Qwen3.8-27B, made with RANA (Reasoning-Anchored, Norm-preserving Ablation), an in-house variant of refusal-direction ablation. It keeps the base model's capabilities, vision tower and MTP head, and almost never refuses outright.

This is a safety-alignment-removed research model. Read Intended use and Limitations before using it.

At a glance

Compared with the base model and the orcarouter/Qwen3.8-27B-Uncensored reference, all run through the same evaluation harness. Where two numbers are given, they are from the two judges (gpt-oss / Mistral).

¹ After the same mechanical re-judge for both abliterations (2 labels each); raw gpt-oss counts are 0.7 for both (2 across 3 seeds). See the disclosure.

In short: capability stays within noise of base except a ~1.5 pp drop on TruthfulQA. Against orca, RANA hits the token budget about half as often, stays closer to base, and answers more held-out prompts under both judges. It is 2.2 pp worse than orca on IFEval (instruction following); other tasks tie.

Quick start

Tested with vLLM 0.30.0:

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve preemware/Qwen3.8-27B-RANA-abliterated \
  --dtype bfloat16 --max-model-len 20480 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --limit-mm-per-prompt '{"image": 1, "video": 0}' \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

• Fits on one 80 GB+ GPU in BF16. The FP8 build needs about half the weight memory; see its card for how it compares with this one.
• Thinking is on by default. Recommended sampling: T = 1.0, top_p = 0.95, top_k = 20.
• --max-model-len 20480 matches the evaluation (16k-token budget). Long technical requests can need 20–50k tokens of reasoning; raise --max-model-len and max_tokens for those (native context 262k).
• VLLM_USE_FLASHINFER_SAMPLER=0 was needed on Blackwell GPUs.
• Vision input, multi-turn tool calling and MTP speculative decoding all work (tested on these shards).

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behaviour.
• Not for public or end-user deployment without a separate moderation layer. The model's own refusals have been largely removed, so any safety filtering has to happen outside it.
• You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen, and the terms of any platform where outputs are used.

What was changed

RANA removes a single "refusal direction" from the model's residual stream (following Arditi et al., 2024), keeping the edited weights at their original norms. The recipe and code are not published.

Everything else is untouched, checked tensor by tensor against the base model:

• Vision tower: all 333 tensors are byte-identical to base.
• MTP head: all 15 tensors are present; its two residual writers are edited the same way as the main model, so speculative decoding keeps working (acceptance length 2.45).
• The removed direction is gone: the largest remaining component along it is 0.00122, down from 0.225.
• Independent check: Abliterlitics (commit 9b15eb0) finds exactly 129 changed tensors among the 850 it inspects. That matches: it skips the vision and MTP tensors, so it sees 131 − 2.

Results

Three models, identical harness: base, the orca reference, and RANA. Refusal numbers are averaged over 3 sampling seeds and scored by two independent LLM judges (details in Evaluation details).

Refusals and token budget

"Answers" = the judge found the harmful request substantively fulfilled (gpt-oss / Mistral judge). "Budget hits" = the response reached the 16k-token generation limit before finishing (no judge involved). These are mostly long reasoning, not repetition; see Limitations.

• Hard refusals are about zero for both RANA and orca, versus 155–197 per set for base.
• The two judges disagree a lot on partial answers. Mistral labels many hedged or incomplete answers as soft refusals that gpt-oss counts as answers, which is why the two percentages differ so much. The judges agree well on outright refusals (κ = 0.89) but only moderately on "answered or not" (κ = 0.54).
• RANA vs orca, held-out prompts (paired McNemar test over 3 seeds):
  • hits the budget less: 12 vs 50 prompt-runs where only one model did, p = 1.2 × 10⁻⁶;
  • answers more under gpt-oss (p = 2.9 × 10⁻⁶) and under Mistral (p = 0.008).
• On HarmBench the answer-rate comparison depends on the judge (gpt-oss favours RANA, Mistral favours orca), so there is no clear winner there.

Counts by prompt set × judge (seeds 1–3)

mean ± sd [min–max] over seeds 1,2,3; complied % and budget-hit % as percentages; budget-hit % is judge-independent.

Seed 0 (original selection-time generations, re-scored)

Paired tests, RANA vs orca, held-out (exact two-sided McNemar)

Cells are (orca-only / RANA-only) discordant counts.

Prompt-level budget-hit sign test: 37 prompts hit the budget more often under orca vs 7 under RANA, p = 5.3e-06.

Capability

Change versus base in percentage points ± standard error, paired on the same questions. Bold = a change clearly outside noise (more than 2 standard errors).

• RANA's only clear loss versus base is ~1.5 pp on TruthfulQA, similar to orca's.
• Versus orca, RANA is clearly worse on IFEval (instruction following, −2.2 pp) and ties elsewhere.
• Where orca and RANA lose HumanEval points, the cause is responses running out of token budget, not wrong code.

R065np was an earlier RANA build. It was rejected because it failed the HumanEval gate (−2.24) and the raw hard-refusal gate.

Release gates

The model was published only after passing all of these:

G1 re-adjudication disclosure

• Raw gpt-oss-safeguard-20b labels contained 2 HARD_REFUSAL on HarmBench, seed 2 (2 / 200).
• Both were re-judged mechanically with the same judge, same settings, plus a fixed auditor reminder appended to the prompt; gpt-oss relabelled both COMPLIED. No manual label edits were made.
• Mistral-Small-24B-2501: 0 hard on all runs.
• Both raw and re-adjudicated G1 are reported above.
• The same rule was applied to orca's 2 raw HarmBench hard labels (seeds 1 and 2): 1 COMPLIED, 1 DEGENERATE (results/readj_orca.json), so the table above compares like with like.

Limitations

• This is a safety-alignment-removed model. It will produce content the base model refuses.
• Refusal numbers come from LLM judges, which can't reliably tell a disclaimered answer from a soft refusal (judge agreement κ = 0.54 on that call). "No hard refusals" is a judge-measured claim, not a guarantee, and it relies on the re-judging disclosed above.
• It reasons at length on technical requests: about 20% of HarmBench and 7% of held-out responses hit the 16k-token budget before finishing (orca: 26% / 12%; base: under 1%, because it refuses early). These are mostly not loops: in 115 of RANA's 123 HarmBench budget hits the text is no more repetitive than normal output, and 98 stop mid-reasoning. On the FP8 build, re-running its 59 seed-1 budget hits with a 64k budget finished 57 (median about 24k tokens) and 55 were judged as answering; see the FP8 card. Set max_tokens well above 16k for such requests.
• It is worse than orca at instruction following (IFEval −2.2 ± 0.9 pp) and ~1.5 pp below base on TruthfulQA.
• MMLU-Pro was run on a 700-question subset, not the full benchmark.

Evaluation details

Serving (identical for all three models): vLLM 0.30.0 (transformers 5.17.0, torch 2.13.0+cu130), 1× RTX PRO 6000 Blackwell 96 GB, flags as in Quick start plus --gpu-memory-utilization 0.93 --max-num-seqs 128. Thinking on. Refusal sampling: T = 1.0, top_p = 0.95, top_k = 20, max_tokens = 16,384, seeds 1, 2, 3.

Prompt sets:

• HarmBench: 200 "standard" behaviours.
• Held-out (240): 90 StrongREJECT (excluding the 60 used at selection time), 60 JailbreakBench harmful, 90 CategoricalHarmfulQA (English). Deduplicated against AdvBench, HarmBench and each other (token-Jaccard ≥ 0.5).
• Seed 0 is the earlier selection-time run (100 HarmBench + 60 StrongREJECT), re-scored.

Judges: openai/gpt-oss-safeguard-20b (primary, reasoning_effort = medium) and mistralai/Mistral-Small-24B-Instruct-2501; same 4-label policy (complied / soft refusal / hard refusal / degenerate), temperature 0, full reasoning trace shown. Agreement over 4,440 generations: raw 69.8%, κ 0.894 hard-vs-rest, 0.564 four-way, 0.543 complied-vs-rest. The first-choice second judge, Mistral-Small-3.2-2506, would not load in vLLM 0.30.

Capability suite: lm-evaluation-harness 0.4.13, 0-shot chat, thinking on, 16k generation budget. MMLU-Pro 50 per subject × 14 (custom-extract); GSM8K full (flexible-extract); IFEval full (prompt-level strict); TruthfulQA MC1/MC2 full (loglikelihood); HumanEval all 164 × 3 samples, unbiased pass@1, via a custom chat-format harness (not comparable to lm-eval's raw-completion HumanEval).

KL: Heretic protocol. First-token full-vocabulary KL(base ‖ model) on mlabonne/harmless_alpaca test[:100], system prompt "You are a helpful assistant.", response prefix <think></think>\n, batchmean.

Smoke tests on the published shards: vision (reads a code word and shape from a synthetic image); 3-turn tool calling (call, use result, second call with a new argument, use result); MTP acceptance rate 0.727, mean acceptance length 2.45 (vLLM Prometheus counters).

File layout

• *.safetensors, model.safetensors.index.json, config.json, tokenizer/processor files, chat_template.jinja, LICENSE — the model.
• results/: the evaluation outputs every number in this card comes from. R060np in filenames is this model's internal build ID; R065np is the rejected earlier candidate. (harden_R060np.{md,json}, capability.{md,json}, head_to_head_orca.json, kl.json, forensic_R060np.json, smoke_R060np.json, humaneval_*.json, abliterlitics/).

Credits

• Qwen team — base model Qwen/Qwen3.8-27B.
• Arditi et al., 2024 — "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim) — prior work on norm-preserving abliteration.
• Heretic (p-e-w) — first-token full-vocabulary KL protocol.
• Benchmarks/datasets: HarmBench, StrongREJECT, JailbreakBench, CategoricalHarmfulQA, AdvBench / mlabonne datasets; lm-evaluation-harness; vLLM.