LMEnt 1B — Baseball RMU+EMBER
This is the paper-selected RMU+EMBER post-training edit for Baseball in Can Concept Erasure Reproduce Concept Exclusion? A Matched Evaluation of EMBER, RMU, and SNMF by Gal Barak, Tamar Tabbach, Itamar Stahl, and Adam Fleisher. It starts from the shared full control; it is compared with the separately trained concept-excluded twin. It was not trained with concept-linked chunks masked from the loss.
Model and method
English OLMo2 1B causal language model, trained on the LMEnt entity-annotated Wikipedia corpus; base model with no instruction tuning. EMBER at δ = 10 was applied first. RMU then ran on that edited model. The selected RMU setting updates MLP down-projections in layers 3–5 with mid steering and retain weight α = 100. RMU used learning rate 1e-4, batch size 1, 150 updates, seed 42, and maximum sequence length 512.
Selected checkpoint
- Appendix B.3 configuration: EMBER delta 10; layer 5, mid steering, alpha 100.
- Candidate label:
rmuember_baseball_L5mid_a100. - Selection was made on the paper's selection split using the fixed rule, before held-out testing.
Evaluation
The table reports the paper's held-out test values; checkpoint selection used separate target, neighboring-topic, and SciQ questions and did not use this twin.
| Metric | Value |
|---|---|
H_test (target efficacy and preservation) | 0.465 |
R_abs (correct-answer NLL distance to twin / full-model distance) | 2.079 |
R_KL (teacher-forced full-vocabulary KL distance to twin / full-model distance) | 2.015 |
For either proximity ratio, values below one indicate movement toward the twin; values above one indicate greater distance than the full control on that measure. Suppression and resemblance to the twin are distinct results. The selected ensemble had low alignment with RMU's target control direction, as discussed in the paper.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "itamarstahl/lment-1b-baseball-rmu-ember-l5mid-a100-b131k"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
Limitations
The paper tests three selected concepts with 50 held-out target questions per concept. These measurements do not establish broad knowledge removal, safety, or generalization to other concepts. This Wikipedia-derived base model may reproduce errors or biases in its training material. No weight license is asserted in this card.
Citation
Gal Barak, Tamar Tabbach, Itamar Stahl, and Adam Fleisher. Can Concept Erasure Reproduce Concept Exclusion? A Matched Evaluation of EMBER, RMU, and SNMF. 2026.