Emese-Ér (517M)
Ér ("brook/rill" — the smallest flowing water in the family name scheme) is the edge/embedded/research tier of the Emese Hungarian model family. Unlike the other three tiers, Ér is not built on EuroLLM — it's a from-scratch Hungarian foundation model, pretrained on ~4.5B tokens of Hungarian text with a custom 32k SentencePiece tokenizer.
| Parameters | 516.8M |
| Base | none — from-scratch pretraining |
| Architecture | LLaMA-style decoder (RoPE + YaRN, RMSNorm, SwiGLU, tied embeddings) |
| Hidden / layers / heads | 1280 / 24 / 20 |
| FFN size | 3456 |
| Vocabulary | 32,000 (custom Hungarian SentencePiece Unigram) |
| Max context length | 32,768 tokens (trained at 1,024, YaRN-extended to 8,192 during CPT, RoPE scale 4.0 rated to 32,768) |
| Precision | bfloat16 |
| License | MIT |
Formats in this release
| Folder | Format | Notes |
|---|---|---|
er/ (this repo) | bf16, MLX-native safetensors | not a standard transformers repo — see below |
er-mlx/ | MLX q8 | mlx_lm-loadable, quantized |
⚠️ Not transformers-compatible out of the box
Ér predates the EuroLLM/Llama-arch pipeline used for Csermely/Patak/Folyó. Its config.json uses a
custom schema (d_model, n_layers, n_heads, d_ff — not HF's hidden_size/num_hidden_layers/...)
and its weight tensor names are custom (embed.weight, layers.N.attn.wq.weight, ...), not the standard
model.embed_tokens.weight / model.layers.N.self_attn.q_proj.weight naming. AutoModelForCausalLM. from_pretrained will fail on this repo. Use the project's own model.py (an MLX Emese class) or load
er-mlx/ via mlx_lm, which does understand this checkpoint despite the non-standard config.
Usage (MLX, via mlx_lm)
from mlx_lm import load, generate
model, tok = load("er-mlx") # or "er" — both load via mlx_lm
print(generate(model, tok, prompt="A magyar nyelv", max_tokens=100))
Usage (project's own loader)
python generate.py --model models-release/er/ --prompt "A magyar nyelv"
Training
- Pretraining (from scratch): ~4.5B tokens of Hungarian text (the same corpus family documented in
corpus/cpt/README.md: Wikipedia + HPLT web text, quality-filtered). No separate "CPT" stage — this is the base pretraining, there is no upstream foundation model underneath it.
Benchmarks
Not applicable — emese-bench (and its Ultimate/BlindSpot predecessors) are chat-instruction
benchmarks (ChatML turns, persona/safety/instruction-following categories); Ér has no chat template or
instruction-tuning, so scoring it on those benchmarks would not be a meaningful comparison against the
other three tiers. During development it was evaluated on HuCoLA (Hungarian grammatical-acceptability
classification) instead, not on chat-style benchmarks.
- No SFT / no DPO. Ér ships as a base/research model only — it has no chat template, no persona training, no instruction-tuning. It is not conversational; treat it as a research/embedding artifact for downstream fine-tuning experiments, not an assistant.
Limitations
- Base model only — expects raw text continuation, not chat-formatted prompts.
- Smallest tier by a wide margin (517M vs. 1.7B+ for the rest of the family) — limited world knowledge and reasoning capacity even relative to Csermely.
- Requires custom loading code (
model.py) for anything beyondmlx_lm.