qwen3.6-35B-A3B-av-1200
RL-tuned Activation Verbalizer (AV) for Qwen/Qwen3.6-35B-A3B — the SFT model
qwen3.6-35B-A3B-av-sft
after 1200 steps of GRPO against a frozen reconstruction reward. Companion reward/
reconstruction model (frozen throughout RL):
qwen3.6-35B-A3B-ar-sft.
RL setup
- GRPO: group size 8, group-normalized advantages, single-update policy gradient, k3 KL penalty (β=0.02) against the SFT policy, temperature 1.0 rollouts
- Reward: cosine similarity between the frozen AR's reconstruction of the generated explanation and the original activation — i.e. the roundtrip metric itself
- LoRA r=32 (attention/GDN/shared-expert), merged into plain HF weights
- Prompts: held-out layer-29 activation vectors, injected at the marker token
- 1200 steps total (200 + 1000 continuation), batch 24 prompts × 8 samples/step
Held-out roundtrip (AV → text → frozen AR → vector, n=128)
| checkpoint | FVE (norm) | cos mean / median | FVE (raw) |
|---|---|---|---|
| SFT (av-sft) | 0.418 | 0.807 / 0.835 | −0.26 |
| GRPO 200 | 0.583 | 0.862 / 0.887 | +0.17 |
| GRPO 1200 (this) | 0.644 | 0.882 / 0.907 | +0.32 |
Usage notes
Same injection contract as the SFT model: nla_meta.yaml + injection_scale.txt
included; scan for the marker token id at inference time (never precompute
positions). See the NLA repo for injection utilities.