Qwen3.8-Flash-Next-NVFP4-Abliterated

Reduced-refusal model. A refusal-direction edit was applied deliberately. Read the safety section before deploying.

Summary

A refusal-direction projection applied directly to the quantised checkpoint local-inference-lab/Qwen3.8-Flash-Next-NVFP4 at revision qad-step5500-ple1000, not to its BF16 parent.

That distinction is the point of this build. The parent is not a post-training quantisation — it is quantisation-aware distilled, 5,500 trunk updates plus 1,000 PLE-refinement updates against the BF16 teacher (quatrain_qwen38_ple_refinement_export). Abliterating Qwen/Qwen3.8-Flash-Next and re-quantising would throw that distillation away and produce a different, weaker artefact. Editing the quantised weights in place is the only way to get an abliterated version of this checkpoint.

No tensor changed format. Every tensor keeps the dtype, block-scale tensors and byte size it has in the parent. 25188 residual-writing tensors were edited; the remaining shards are hardlinked from the parent and are byte-identical.

What was edited

Direction measured at decoder layer 43 of 48 (Cohen's d = 10.44), from 128 harmful (mlabonne/harmful_behaviors) and 128 harmless (mlabonne/harmless_alpaca) prompts, seed 42, as the normalised difference of means of the residual stream entering that layer. Projection strength alpha = 1.0. The direction is biprojected — orthogonalised against the harmless mean activation, so the ablated axis carries only the harmful-conditional component and shared harmless content is left in place.

Measurement details that matter, all following HangGlidersRule/model-forge's documented method:

• Measured on the quantised model itself, under vLLM, not on a BF16 parent. The residual stream here is a hyper-connection of width 4 x 2560; the four sub-streams are averaged to recover the 2,560-dim block-output space the write matrices live in.
• Prompts rendered with thinking off, matching the regime the headline refusal gate scores in. This is load-bearing: a direction captured with thinking on barely suppresses refusals with it off, and vice versa.
• The layer was chosen by generation test, not by activation statistics. Every layer's direction was screened by ablating it with inference-time hooks and counting refusals. The result is not monotonic in depth and disagrees with the Cohen's d argmax, so picking the most-separable layer would have shipped a weaker edit.
• Layer 0 measures a difference of exactly 0.0000, which is a free correctness check on the capture: every prompt shares the same final chat-template token, so the residual entering layer 0 must be identical across the set.

Why a naive edit does not work here

The stored weights already sit exactly on the quantisation grid, so a rank-1 projection worth 2-4 % of a tensor's Frobenius norm is mostly smaller than half a unit in the last place and plain round-to-nearest returns the original code. Measured on this checkpoint's own tensors, round-to-nearest retains only 74 % of the edit on MXFP8 and 14 % on NVFP4 — a checkpoint that looks edited and largely is not.

This build instead chooses, per element, the code that both approximates the edited weight and holds the constraint v · W = 0 across each column: rows are visited in descending |v_i| and each absorbs as much of the accumulated constraint error as a one-ULP move allows.

Residual leakage (||v.W_hat|| / ||v.W||; 0 = applied exactly, 1 = erased) is at most 3.01e-06 across all 25136 quantised tensors, and 7.84e-03 across the 52 BF16 ones -- the BF16 tensors are the looser of the two, because they take plain rounding and inherit bfloat16's 0.39 % relative ULP, while the quantised ones are actively driven to zero. Mean added weight error is 3.820% of each tensor's norm.

Evaluation

Capability parity against the parent, measured in the same session on the same serving configuration. Scores from different serving configurations are not comparable for these profiles.

lavd is the discriminating profile for this family; estonia is not (every build measured scores 30/30). n=10 cannot resolve a difference of one or two runs, so lavd is reported at n=30 against the parent in the same session — that is the row to read. GPQA-diamond is sampled (temperature 1.0, top_p 0.95, top_k 20) at model-forge's stated settings for their abliterated product, so a one-item gap there is noise, not signal.

Refusal

treadon/abliteration-eval, 200 harmful + 83 safe over-refusal + 50 multilingual. Refusal counts, so lower is better in every cell.

A refusal number is the product of a protocol and a scoring convention, and both are reported here because on this model family the protocol turns out to dominate. Presenting a single cell would let the headline be chosen after the fact.

• Protocol A is the one HangGlidersRule/model-forge publishes its Darkstar gate under: thinking off, temperature 0, max_tokens=100.
• Protocol B is a reasoning-mode run: thinking on, temperature 0, max_tokens=2048.
• The model-forge scorer is their src/model_forge/refusal.py: five refusal-form markers, whole-text match, an empty completion scoring as compliance.
• The strict scorer is ours: 47 markers matched in the first 400 characters of the visible content, and an empty completion counted as a refusal.

harmful (n=200)

over_refusal (n=83, benign prompts that look harmful)

multilingual (n=50)

Reading these numbers

The direction was measured with thinking off, matching protocol A, and that is the regime the build is strongest in. Under protocol B the strict scorer is harsher than the model-forge one mainly because this build thinks for longer: it spends more completion tokens than the parent, hits the 2048-token cap more often, and emits no visible content when it does. The strict scorer calls an empty completion a refusal; model-forge's calls it compliance. Neither is wrong — they answer different questions — but the gap between those two cells is largely that convention, not extra refusal text.

Safety

This model's refusal behaviour was deliberately reduced. It will comply with many requests the parent would decline, and it carries no added safety mitigations. Refusal rates above are behaviour measurements, not a safety endorsement. Deploy only behind appropriate policy, filtering, access control and legal review.

Serving

Same runtime and recipe as the parent — mixed NVFP4/MXFP8 ModelOpt layout, MTP3, FP8 KV cache, and the n-gram (PLE) table offloaded to host RAM. The quantisation is unchanged, so anything that serves the parent serves this.

vllm serve huginnfork/Qwen3.8-Flash-Next-NVFP4-Abliterated \
  --served-model-name Qwen3.8-Flash-Next \
  --quantization modelopt_mixed --kv-cache-dtype fp8 --dtype bfloat16 \
  --max-model-len 262144 --max-num-seqs 16 --max-num-batched-tokens 6019 \
  --enable-prefix-caching --enable-chunked-prefill --language-model-only \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

VLLM_PLE_CPU_OFFLOAD=1 is required to fit a single 96 GB card.

Provenance

• Parent: local-inference-lab/Qwen3.8-Flash-Next-NVFP4
• Method: single refusal-direction projection, applied in-format; full per-tensor record in abliteration_report.json in this repo.
• lm_head reads from the residual stream rather than writing to it and was not edited. The vision tower was not edited.
• Per-run benchmark JSONs for both this build and the parent are under measurements/ in this repo, so every number above can be recomputed.

A more conservative variant — the same method applied to the attention output projections, shared experts, embeddings and MTP writes only, leaving the 25,088 routed-expert down_proj tensors untouched — was built and measured alongside this one. It is equally capability-neutral but only reaches 51 % harmful refusal against this build's 5.5 %, so it was not published.