SAEVerbalizer-27B
SAEVerbalizer-27B is the default verbalizer from
SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization.
It is initialized from google/gemma-3-27b-it and fine-tuned to generate
natural-language explanations from injected Sparse Autoencoder (SAE) decoder
directions.
Resources
- Paper: arXiv:2608.13538
- Code:
THU-KEG/SAEVerbalizer - Evaluation data:
THU-KEG/SAEVerbalizer-Data - SAE:
google/gemma-scope-2-27b-it
Usage
Run this checkpoint with the inference pipeline provided in the official code repository. The pipeline loads the corresponding SAE decoder direction and injects it into the model's residual stream before generation.
Configuration
- Verbalizer backbone:
google/gemma-3-27b-it - Injection layer: 16
- Training pairs: 48k
- Injection: norm-matched additive injection
- Injection coefficient:
alpha = 0.2 - Injection span: final four tokens of the fixed verbalization prompt
This repository contains a full Transformers checkpoint. Inference also
requires the layer-16 width-262k l0_medium SAE linked above.
License
SAEVerbalizer-27B is a modified Model Derivative of
google/gemma-3-27b-it.
The model weights were modified through SAEVerbalizer fine-tuning and are
distributed under the Gemma Terms of Use.
See the NOTICE file for the required attribution notice.
Citation
@article{meng2026saeverbalizer,
title = {SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization},
author = {Meng, Weihan and Guo, Hongzhu and Jing, Yi and Liu, Dewen and Yao, Zijun and Wang, Xiaozhi and Hou, Lei and Li, Juanzi},
journal = {arXiv preprint arXiv:2608.13538},
year = {2026}
}