Emender E97 1.3B
Emender E97 is a 1.287B-parameter recurrent base language model with nonlinear matrix state and split erase/write control inspired by Gated DeltaNet 2. It is an E97 split-edit model, not the repository's separate GDN2 control.
This revision contains the checkpoint after 513,013,841,920 training tokens
(step 2322520, recorded loss 2.2798). The default repository revision is
the later 513B authority; the 150B authority is retained as a historical
reproducibility revision.
Artifact
- Source checkpoint SHA-256:
e559df3e8c540aef59ce8c9d73338f255cbe2fb9c7301ab45c7ef36a5b0fb857 - Export: evaluated ScheduleFree train/y weights only
- Tokenizer:
p50k_base - Context used in training: 2,048 tokens
- Training approach: local DiLoCo with periodic model averaging
- Raw optimizer state and pickle checkpoint: not included
The model is a raw base LM, not an instruction or chat model. A full HellaSwag
audit of the 513B authority scored 0.3651663 normalized accuracy; GPT-2 XL in
the same harness scored 0.4890460.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("spinozans/emender-e97-1.3b")
model = AutoModelForCausalLM.from_pretrained(
"spinozans/emender-e97-1.3b", trust_remote_code=True, dtype="auto"
)
The custom loader requires the matching Emender source package:
pip install "git+https://github.com/spinozans/emender.git@2b5d135587cca9d1081dbbdf7b35a8ac2629db5d"
The qualified fused path targets AMD ROCm/Frontier. The bundled Transformers wrapper defaults to the slower native portable path.
Limitations and license
This is a research checkpoint with limited benchmark coverage. It can produce
incorrect, biased, or unsafe text. No standalone model license has yet been
selected; license: other is intentional and no permissive license should be
inferred from repository visibility.