Qwen3.8-27B-heretic-r1n

Iterative true rank-1 (PRE) Heretic of Qwen/Qwen3.8-27B. After each accepted round the residual direction r is recomputed on the updated weights, then Heretic is run again. Cumulative KL vs the original model is capped at 0.1.

对 Qwen/Qwen3.8-27B 做 真 rank-1（PRE） 迭代 Heretic：每轮接受后再在新权重上重算残差方向 r，然后进入下一轮。相对原模型的累计 KL 上限为 0.1。

Model

Method

Heretic is training-free directional ablation (Arditi et al., 2024). It estimates a per-layer residual direction r from the first generated token, then writes a low-rank update into the output projections so that component is suppressed. Optuna TPE searches ablation weights to lower refusal-keyword hits while keeping KL to the parent model small.

Why PRE (true rank-1). Heretic row-normalization has three modes:

• none: ΔW = -λ v (vᵀ W) — rank-1, no row-norm handling
• pre: same rank-1 form, but computed on row-normalized W, then scaled back by the original row norms: ΔW = -λ v (vᵀ W_norm) ⊙ ‖W‖_row
• full: apply the PRE update, renormalize rows to restore magnitudes, then keep an SVD approximation of rank full_normalization_lora_rank (default 3). That is not rank-1.

This release uses PRE only. Each accepted round is therefore one true rank-1 update. Two accepted rounds are two successive rank-1 updates, not a single rank-2 factorization.

Why iterate, and why recompute r. After a merge the residual geometry changes, so the next direction is measured on the new weights. Reusing r from round 1 would not be a fresh rank-1 step.

KL accounting. Search-time KL in a Heretic log is vs the current parent (incremental). Acceptance always re-evaluates vs the original base with --evaluate-model. Only that cumulative KL is constrained to ≤ 0.1.

Thinking-model cut. Qwen3.8 thinks by default. Residuals and keyword scoring are taken at the start of the visible answer, with --response-prefix $'\n</think>\n\n'. Do not put that prefix into the chat template at inference time.

Pipeline

1. Start from official Qwen/Qwen3.8-27B (not from a previous heretic checkpoint).
2. Round 1: PRE, 200 trials / 60 startup → accept trial 162.
3. Recompute r on the round-1 merge.
4. Round 2: PRE, 120 trials / 40 startup → accept trial 109 (this repo).
5. Recompute r, try round 3 → reject (cumulative KL 0.1983 > 0.1).

Results vs original

Heretic default English eval: keyword refusals / 100 prompts, and KL on the harmless split.

Round 3 would have lowered keywords further, but broke the KL cap. The shipped weights stay at round 2.

Delivered parameters (round 2, on top of round 1)

Per-layer directions. Ablation strength is a piecewise-linear kernel over layer index.

Round 1 (already baked in): trial 162, attn.o_proj max 0.90 @ 53.74, mlp.down_proj max 1.31 @ 62.63.

Usage

Needs a Transformers build that loads Qwen3.8 (qwen3_5 / Qwen3_5ForConditionalGeneration), e.g. Transformers 5.15+. Same chat template and sampling defaults as the base.

from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "YOUR_USER/Qwen3.8-27B-heretic-r1n"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="auto", device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain residual connections in one paragraph."},
]
prompt = tok.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,  # True (default) keeps <think>…</think>
)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.8,
    top_k=20,
    presence_penalty=1.5,
)
print(tok.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

• Instruct / non-thinking: enable_thinking=False, temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5
• Thinking: enable_thinking=True (default), temperature=1.0, top_p=0.95, top_k=20; optional reasoning_effort in {xhigh, medium, low}
• Image / video I/O is unchanged from the base; only language-model output projections were modified

Reproduce the procedure

Exact trial indices depend on the Optuna study. The procedure is:

pip install -U heretic-llm
PREFIX=$'\n</think>\n\n'

heretic --model Qwen/Qwen3.8-27B \
  --response-prefix "$PREFIX" --row-normalization PRE \
  --n-trials 200 --n-startup-trials 60 \
  --model-action save --export-strategy merge \
  --save-directory ./r1

# recompute r on the new weights, then search again
heretic --model ./r1 \
  --response-prefix "$PREFIX" --row-normalization PRE \
  --n-trials 120 --n-startup-trials 40 \
  --model-action save --export-strategy merge \
  --save-directory ./r2

# cumulative KL must be scored against the original base
heretic --model Qwen/Qwen3.8-27B \
  --evaluate-model ./r2 --response-prefix "$PREFIX"

Notes

• Cumulative KL 0.0931 is close to the 0.1 cap; another PRE round measured 0.1983 and was discarded.
• Keyword counts are English-only (Heretic default markers / eval set).
• Vision encoder and MTP layers were not ablated.

中文说明

在官方 Qwen3.8-27B 上用 Heretic 1.4.0 做迭代真 rank-1 消融。PRE 在行归一化后的 W 上写 ΔW = -λ v (vᵀ W_norm)，再乘回行范数，每轮严格 rank-1。FULL 会再做行范数还原并用默认 rank-3 的 SVD 近似，因此不是本仓库的做法。

每轮接受后必须在新权重上重算残差方向 r，不能复用上一轮的 r。搜索日志里的 KL 是相对本轮父模型的增量；验收一律对原模型跑 --evaluate-model，累计 KL 不得超过 0.1。Qwen3.8 默认带思考，取点和打分都切在 \n</think>\n\n 之后，推理时不要把这个 prefix 写进模板。

交付点是第 2 轮 trial 109：Keyword 18 / 100，累计 KL 0.0931。第 3 轮 Keyword 可到 8，但 KL 0.1983 超标，未合并。视觉塔和 MTP 未改。权重为 BF16 merge，12 个分片。许可 Apache-2.0。