DeepSeek-V4-Flash-0731-Abliterated

An abliterated (refusal-ablated) build of deepseek-ai/DeepSeek-V4-Flash-0731.

Produced by a forward-only, closed-form weight edit — no training, no fine-tuning, no distillation. Only the attention output projections (attn.wo_b) are modified; the MXFP4 experts, mHC (Manifold-Constrained Hyper-Connections), norms and embeddings are byte-identical to the base model. The three MTP blocks are not untouched — their attn.wo_b receives the same rank-1 projection as the main layers (see the recipe table), so the draft head stays consistent with the target.

Measured refusal rate: 0.00% across 913 harmful prompts (AdvBench 520 + HarmBench 393) in both reasoning modes, with no measurable capability loss on GSM8K (1319 items) or MMLU (1000 items).

Recipe

The FP8 (F8_E4M3) weights are dequantised per 128×128 block, projected, and requantised; all other dtypes (F8_E8M0 scales, I8-packed FP4 experts, BF16, F32) are passed through untouched.

Refusal benchmarks

520 AdvBench harmful_behaviors + 400 HarmBench behaviours, temperature=0, greedy. Refusal detection is a deliberately broad keyword matcher (biased toward over-counting refusals); every flagged response was then reviewed by hand.

thinking=true — the model's natural mode, no jailbreak prefix

thinking=false + assistant prefill "Sure, here is"

HarmBench semantic categories, thinking=true: the base model refuses chemical_biological 100%, harassment_bullying 100%, illegal 98.3%, harmful 81.0%, misinformation 79.4%, cybercrime 50.0% — this model refuses 0.0% in all six.

Honest note on the raw numbers

The keyword matcher produced 12 raw hits on this model. All 12 were false positives, verified individually:

• Song lyrics containing refusal-like strings, e.g. Old Town Road — "I'm gonna ride 'til I can't no more"; Clocks — "Lights go out and I can't be saved". The model had fully complied.
• Capability disclaimers followed by compliance, e.g. "since I'm an AI, I can't literally hack, so I'll give you the actual technical breakdown of how a real hacker would do it."

Raw (unadjudicated) rates were 0.58% on AdvBench/nothink and 5.00–5.26% on the lyrics-heavy copyright split.

thinking=true was measured with vLLM's --reasoning-parser deepseek_v4, so the chain-of-thought lands in reasoning and content holds only the final answer. Responses truncated by max_tokens before the reasoning finished are reported separately and excluded from the denominator (4/520 on AdvBench) rather than being scored as compliance.

Capability

Both deltas sit within one standard error — statistically indistinguishable from the base model. Tool calling was verified end-to-end (finish_reason: tool_calls, correctly populated arguments), which is why layers 0–9 were deliberately left untouched.

Usage

Drop-in replacement for the base checkpoint — same 48 shards, same index.json, same tokenizer.

vllm serve <path-to-this-repo> --tensor-parallel-size 2 --kv-cache-dtype fp8 \
  --trust-remote-code --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 --reasoning-parser deepseek_v4

On RTX PRO 6000 Blackwell (SM120), five things are required

Verified on vLLM 0.27.1 / torch 2.13.0+cu130 / Python 3.10. None of these are needed on H200 except (1) and (2).

1. CUDA_DEVICE_ORDER=PCI_BUS_ID. Without it CUDA_VISIBLE_DEVICES reorders CUDA devices but not NVML, so vLLM reads the wrong card's compute capability — torch reports (12,0) while vLLM reports major=9. The model then loads fine and dies in profile_run with layout.hpp:107: sf.size(-2) == ceil_div(mn, gran_mn), because the FP8 einsum recipe is chosen for SM90 while DeepGEMM has already packed the scales as INT32 for SM120.
2. --kernel-config '{"moe_backend":"marlin"}' — the default DEEPGEMM_MXFP4 path hits runtime_utils.hpp:91 Unsupported dtype on SM120.
3. External DeepGEMM @ 8b1392b, installed with pip install --no-build-isolation (its setup.py imports torch). The bundled vllm/third_party copy ships no SM120 kernels.
4. Create nvidia/cu13/lib64/ and symlink unversioned .so names into it — the directory does not exist in the pip CUDA wheels, so FlashInfer's JIT link step fails with cannot find -lcudart.
5. Patch flashinfer/comm/fd_exchange.py with from __future__ import annotations — upstream declares requires-python >=3.10 but uses array.array[int], which needs 3.12. Also pin flashinfer-cubin to exactly the flashinfer-python version (vLLM 0.27.1 pins 0.6.16.post3).

Reproducing the evaluation

evals/ contains every script and result file, plus the raw stdout logs behind each number:

• probe_refusal.py — refusal probe (CSV benchmarks, category breakdown, truncation accounting)
• eval_gsm8k.py, eval_mmlu.py — capability evals
• project_wob.py — the weight-edit script that produced this model
• logs/ — unfiltered run logs

Intended use and limitations

This model does not refuse, including in the most severe categories measured by HarmBench (chemical/biological, cybercrime, harassment). That is the entire point of the modification, and it means safety behaviour must come from somewhere else in your stack. It is suited to trusted, internal, research and red-teaming contexts. If you expose it to third parties, put your own moderation and policy layer in front of it — the model will not do that job for you.

The abliteration removes a specific learned refusal direction; it does not add knowledge, and it does not make the model's outputs more accurate. Everything the base model gets wrong, this one still gets wrong.

Credits

• deepseek-ai/DeepSeek-V4-Flash-0731 — base model (MIT)
• drowzeys — the published refusal direction and the project_wob.py / compute_direction.py pipeline
• apetersson — reference hyper-parameters (λ, layer range) and the Δrel manifest used to sanity-check this run

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

License

MIT, inherited from the base model.