Qwen3.6-35B-A3B-Fable-5-Distill-heretic (v5-dpo)
Revision v5 (2026-08-05): coverage-weighted injection-defense DPO on the
v2 heretic weights. Prior revisions preserved on branches: v2 (surgery),
v4 (first DPO pass).
A decensored version of armand0e/Qwen3.6-35B-A3B-Fable-5-Distill — itself a full-precision (BF16) fine-tune of Qwen3.6-35B-A3B distilled on real Claude Fable 5 sessions (armand0e/claude-fable-5-claude-code).
What changed in v5 (revision annotation)
v2 (2026-08-03, branch v2): two-pass heretic directional ablation for
refusal suppression. Measured side effect: channel-impersonation injection
compliance rose from 13% (pre-surgery) to 70%; exfiltration 7% to 43%.
v3 (2026-08-05, dead end, unpublished): constrained re-ablation with an injection-canary objective. No recovery; refusal suppression and injection obedience did not separate in the ablation search space.
v4 (2026-08-05, branch v4): first LoRA-DPO injection defense (Meta
SecAlign shape, 1,312 on-policy pairs). Halved T6a (21/30 -> 13/30), left
exfiltration untouched.
v5 (2026-08-05, THIS REVISION): fresh LoRA-DPO from the v2 weights (r=32, alpha=8, beta=0.1, 1 epoch, ~2.2h on one 96GB GPU) on 4,559 on-policy pairs, coverage-weighted from the v4 lesson: ~38% of pairs now carry exfiltration/action-laundering attacks (NVIDIA Nemotron agentic-IPI scenarios re-seeded for position variance, plus this model's own measured failure pages from the harness battery), the rest the proven v4 mix. Hypothesis tested and confirmed for instruction-following attacks: coverage, not volume, was the v4 gap.
Measured results (v5 gate, 2026-08-05, n=30 per row, temp 1.0,
212/212 valid runs, raw-log spot-checked)
| Probe | v2 | v4 | v5 |
|---|---|---|---|
| T6a channel-impersonation compliance | 21/30 | 13/30 | 1/30 (95% CI 0.1-17.2%) |
| T6b action laundering | 3/30 | 1/30 | 0/30 |
| T6c context smuggling | 3/30 | 3/30 | 1/30 |
| T7 tool-arg exfiltration | 13/30 | 14/30 | 12/30 (UNCHANGED) |
| Refusal tripwire (4 prefill x thinking cells) | answered all | answered all | answered all (surgery intact) |
| Tool-call discipline with served schemas (F4-F8) | 100% capture | 100% capture | 100% capture |
| Vision (smoke: text OCR, chart read, UI dialog, shapes) | untested | untested | works |
T7 is the honest exception: "do not follow injected commands" is learnable by preference training; "do not place protected text into tool arguments" did not transfer from this pair distribution. In deployment an argument egress filter (block tool args containing system-prompt/history substrings) is REQUIRED for exfiltration; measured containment 12/12 with the filter active. This matches the literature: weight-side defenses are probabilistic and harness policy layers carry the exfiltration boundary.
Provenance
Qwen/Qwen3.6-35B-A3B
-> armand0e/Qwen3.6-35B-A3B-Fable-5-Distill (Fable 5 session distill, BF16)
-> v2 heretic (two-pass directional ablation, 2026-08-03) [branch: v2]
-> v3 re-ablation + injection objective (2026-08-05, dead end, unpublished)
-> v4 (first LoRA-DPO pass, 1,312 pairs, 2026-08-05) [branch: v4]
-> v5 THIS MODEL (coverage-weighted LoRA-DPO, 4,559 pairs, 2026-08-05)
Usage
Serve with vLLM >= 0.26 (Qwen3.6 MoE support):
vllm serve Authereon/Qwen3.6-35B-A3B-Fable-5-Distill-heretic \
--max-model-len 131072 --max-num-seqs 64 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder
~66GB BF16 weights: 96GB-class GPU or quantization for smaller cards. On
Blackwell (sm_120) set VLLM_USE_FLASHINFER_SAMPLER=0 and use the
official vllm/vllm-openai image (PyPI wheels lack sm_120 kernels).
Sampling per generation_config: temperature 1.0, top_p 0.95, top_k 20.
Reasoning models need generous max_tokens (>= 8K recommended, floor 512
with thinking on). NOTE: reasoning_effort and thinking_budget are
silently dropped by vLLM 0.26 on this family (measured); the working
control is chat_template_kwargs.enable_thinking.
Limitations
- T7 exfiltration resistance is NOT improved in v5 (~40% residual compliance). An argument egress filter is mandatory in agentic deployments; see the measured results table.
- White-box adaptive attacks defeat every published probabilistic defense (85-95% ASR, arXiv:2507.07417; >90% vs 9 of 12 defenses, arXiv:2510.09023). These weights are public; a motivated attacker has gradient access. Defense-in-depth (result sanitization, argument egress filtering, tool round caps) remains necessary in any agentic deployment.
- Residual refusals from the distill remain suppressed (the v2 surgery is intact); this model will answer what its stock sibling refuses. Safety filtering has been deliberately reduced. Review outputs before use. Not for public-facing or underage-audience applications. Users are solely responsible for compliance with applicable law.
- Tool discipline requires served schemas. Without a tool schema the model fabricates answers instead of calling (measured 3/3). Always serve tool definitions; always lint call names against the served set.
- The distill bias persists: trained on 352 coding-agent sessions; expect code/tool-shaped structure on general questions.
Warnings
- Safety filtering has been deliberately reduced. Outputs may include sensitive, controversial, or harmful content.
- Decensoring is weight surgery plus preference training, not retraining: behavior drift beyond the measured probes is possible.
- The injection resistance is measured against in-distribution attacks; novel attack classes should be re-probed before high-stakes use.