svdsafety_l2_addbase_top256
A safety-ablation research artifact, not a chat model. Grafts the leading rank-256 subspace of dW onto the base (non-chat) model: W' = W_base + U_k S_k V_k^T. Tests whether that slice alone carries refusal behaviour.
Built from meta-llama/Llama-2-7b-chat-hf and meta-llama/Llama-2-7b-hf with
compress/build_delta_ablation.py, k=256, all 224 projections edited, seed 42.
The weight perturbation ||W' - W||_F / ||W||_F is 0.03719 averaged over
matrices.
Measured behaviour
AdvBench (520 prompts), greedy chat decoding, judged by
cais/HarmBench-Llama-2-13b-cls:
| this checkpoint | dense Llama-2-7b-chat | |
|---|---|---|
| attack success rate | 0.0077 | 0.0019 |
| refusal rate (harmful) | 0.996 | 0.994 |
| degenerate output rate | 0.006 | 0.000 |
Why these exist
Compressing a chat model by singular value discards its alignment update almost
by construction: sigma(dW) is orders of magnitude below sigma(W_base), and
the leading 64 directions of dW hold only 15% of its energy. These checkpoints
test whether that subspace nevertheless carries safety, by deleting it and
comparing against controls matched on exactly the same amount of damage:
| edit (k=64; perturbation 0.02423 for all three) | ASR | refusal |
|---|---|---|
| remove top-64 of dW | 0.0962 | 0.831 |
| remove bottom-64 of dW | 0.0038 | 0.994 |
| remove random 64 of dW | 0.0058 | 0.992 |
Only the top-64 costs refusal, which is what justifies protecting that subspace
by reserved rank rather than by singular value. The subspaces themselves are at
Jeesup/svdsafety_l2_delta_basis.
Intended use
Measuring how alignment survives low-rank compression. Do not deploy these as assistants: they are modified specifically to change refusal behaviour, and several are degraded in general capability as well.
Licence
Llama 2 Community License. LICENSE.txt and USE_POLICY.md are included here
and use of this derivative is bound by both. Built with Llama 2.