LMEnt 1B — Baseball EMBER
This is the paper-selected EMBER post-training edit for Baseball in Can Concept Erasure Reproduce Concept Exclusion? A Matched Evaluation of EMBER, RMU, and SNMF by Gal Barak, Tamar Tabbach, Itamar Stahl, and Adam Fleisher. It starts from the shared full control; it is compared with the separately trained concept-excluded twin. It was not trained with concept-linked chunks masked from the loss.
Model and method
English OLMo2 1B causal language model, trained on the LMEnt entity-annotated Wikipedia corpus; base model with no instruction tuning. EMBER edits selected input-embedding directions at strength δ = 10. Its semantic features were factorized at rank 100 from 300 target and 300 neutral sentences (sparsity 0.02, seed 44); a semantic judge retained target-related features at confidence at least 0.85.
Selected checkpoint
- Appendix B.3 configuration: delta 10.
- Candidate label:
ember_baseball_d10. - Selection was made on the paper's selection split using the fixed rule, before held-out testing.
Evaluation
The table reports the paper's held-out test values; checkpoint selection used separate target, neighboring-topic, and SciQ questions and did not use this twin.
| Metric | Value |
|---|---|
H_test (target efficacy and preservation) | 0.639 |
R_abs (correct-answer NLL distance to twin / full-model distance) | 2.128 |
R_KL (teacher-forced full-vocabulary KL distance to twin / full-model distance) | 2.082 |
For either proximity ratio, values below one indicate movement toward the twin; values above one indicate greater distance than the full control on that measure. Suppression and resemblance to the twin are distinct results.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "itamarstahl/lment-1b-baseball-ember-d10-b131k"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
Limitations
The paper tests three selected concepts with 50 held-out target questions per concept. These measurements do not establish broad knowledge removal, safety, or generalization to other concepts. This Wikipedia-derived base model may reproduce errors or biases in its training material. No weight license is asserted in this card.
Citation
Gal Barak, Tamar Tabbach, Itamar Stahl, and Adam Fleisher. Can Concept Erasure Reproduce Concept Exclusion? A Matched Evaluation of EMBER, RMU, and SNMF. 2026.