LMEnt 1B — Baseball SNMF
This is the paper-selected SNMF post-training edit for Baseball in Can Concept Erasure Reproduce Concept Exclusion? A Matched Evaluation of EMBER, RMU, and SNMF by Gal Barak, Tamar Tabbach, Itamar Stahl, and Adam Fleisher. It starts from the shared full control; it is compared with the separately trained concept-excluded twin. It was not trained with concept-linked chunks masked from the loss.
Model and method
English OLMo2 1B causal language model, trained on the LMEnt entity-annotated Wikipedia corpus; base model with no instruction tuning. SNMF was applied directly to the completed full control. The selected SNMF setting uses ratio-based feature selection and edits the MLP input and output weights. SNMF factorized layers 4–6 at rank 100 with ratio threshold 2.0, seed 42, maximum sequence length 256, and exact component removal strength 1.
Selected checkpoint
- Appendix B.3 configuration: ratio selection, both sides.
- Candidate label:
snmf_baseball_ratio_both. - Selection was made on the paper's selection split using the fixed rule, before held-out testing.
Evaluation
The table reports the paper's held-out test values; checkpoint selection used separate target, neighboring-topic, and SciQ questions and did not use this twin.
| Metric | Value |
|---|---|
H_test (target efficacy and preservation) | 0.129 |
R_abs (correct-answer NLL distance to twin / full-model distance) | 0.967 |
R_KL (teacher-forced full-vocabulary KL distance to twin / full-model distance) | 0.962 |
For either proximity ratio, values below one indicate movement toward the twin; values above one indicate greater distance than the full control on that measure. Suppression and resemblance to the twin are distinct results. All candidates for this method and concept had zero selection efficacy; the fixed tie-break rule determined this choice.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "itamarstahl/lment-1b-baseball-snmf-ratio-both-b131k"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
Limitations
The paper tests three selected concepts with 50 held-out target questions per concept. These measurements do not establish broad knowledge removal, safety, or generalization to other concepts. This Wikipedia-derived base model may reproduce errors or biases in its training material. No weight license is asserted in this card.
Citation
Gal Barak, Tamar Tabbach, Itamar Stahl, and Adam Fleisher. Can Concept Erasure Reproduce Concept Exclusion? A Matched Evaluation of EMBER, RMU, and SNMF. 2026.