svdsafety_l2_ablate_top16
A safety-ablation research artifact, not a chat model. Removes the leading rank-16 subspace of dW = W_chat - W_base from every projection of the chat model: W' = W_chat - U_k S_k V_k^T.
Built from meta-llama/Llama-2-7b-chat-hf and meta-llama/Llama-2-7b-hf with
compress/build_delta_ablation.py, k=16, all 224 projections edited, seed 42.
The weight perturbation ||W' - W||_F / ||W||_F is 0.01499 averaged over
matrices.
Measured behaviour
AdvBench (520 prompts), greedy chat decoding, judged by
cais/HarmBench-Llama-2-13b-cls:
| this checkpoint | dense Llama-2-7b-chat | |
|---|---|---|
| attack success rate | 0.0154 | 0.0019 |
| refusal rate (harmful) | 0.979 | 0.994 |
| degenerate output rate | 0.008 | 0.000 |
Why these exist
Compressing a chat model by singular value discards its alignment update almost
by construction: sigma(dW) is orders of magnitude below sigma(W_base), and
the leading 64 directions of dW hold only 15% of its energy. These checkpoints
test whether that subspace nevertheless carries safety, by deleting it and
comparing against controls matched on exactly the same amount of damage:
| edit (k=64; perturbation 0.02423 for all three) | ASR | refusal |
|---|---|---|
| remove top-64 of dW | 0.0962 | 0.831 |
| remove bottom-64 of dW | 0.0038 | 0.994 |
| remove random 64 of dW | 0.0058 | 0.992 |
Only the top-64 costs refusal, which is what justifies protecting that subspace
by reserved rank rather than by singular value. The subspaces themselves are at
Jeesup/svdsafety_l2_delta_basis.
Intended use
Measuring how alignment survives low-rank compression. Do not deploy these as assistants: they are modified specifically to change refusal behaviour, and several are degraded in general capability as well.
Licence
Llama 2 Community License. LICENSE.txt and USE_POLICY.md are included here
and use of this derivative is bound by both. Built with Llama 2.