eo-mt-v13-tiny-bidir
MarianMT bidirectional English<->Esperanto, ~69M params (mt/configs/tiny.yaml).
Fresh-training baseline on the new v13 corpus, which drops the OPUS-100
KDE/GNOME/Ubuntu contamination that caused the v12 UI-label collapse
failure mode (@ info: whatsthis -> & Resize, Name -> A tool to store a new image).
Config (tiny.yaml)
d_model 512, 6 enc + 6 dec layers, 8 heads, ffn 2048. Total: 69.2M / trainable 68.7M params.
Data
jensjepsen/esperanto-mt-parallel-v13(5.03M rows; no opus-100)jensjepsen/esperanto-mt-math-parallel-v2(sentences + rows)jensjepsen/esperanto-mt-yago-parallel-v2(labels + comments)jensjepsen/esperanto-orca-math-gemini-parallel
Total pool: 5.84M rows/epoch.
Training
Single RTX 5090, bf16, 3 epochs (136,962 steps), batch_size=128 (no grad accum), peak LR 5e-4, cosine decay, warmup 2000 steps, weight_decay 0.01, label smoothing 0.1, dropout 0.1, direction=bidir, val-direction=en2eo. Runtime 4h 32m.
Tokenizer: spm_eneo_48k_v3.model (SPM unigram, case-preserving
nmt_nfkc).
Final val BLEU (cased sacrebleu, ep 3.0)
| set | BLEU |
|---|---|
| FLORES devtest (en2eo) | 28.05 |
| opus100 validation | 32.39 |
| MMLU STEM | 66.79 |
| MMLU full | 58.80 |
| SciQ | 63.22 |
| Math WP | 99.64 |
vs eo-mt-v12-bidir (large 285M, opus-100 contaminated): 95-100% of
v12's BLEU across all sets, with 4.5x fewer params, and cleanly free of
the KDE-junk regression on capitalized-fragment inputs.
Loading
from transformers import MarianMTModel
import sentencepiece as spm
model = MarianMTModel.from_pretrained("jensjepsen/eo-mt-v13-tiny-bidir")
sp = spm.SentencePieceProcessor(model_file="spm_eneo_48k_v3.model")
# Prepend <eo> (id=48001) to input for EN->EO, or <en> (id=48000) for EO->EN.