VEXMLM — Tigrinya NER
Tigrinya token classification fine-tuned from
Hailay/VEXMLM, the vocabulary-extended
XLM-R for Ge'ez-script languages.
Official implementation: https://github.com/hailaykidu/VEXMLM
| Task | token-classification |
| Dataset | Tigrinya NER |
| Language | Tigrinya |
| Architecture | XLMRobertaForTokenClassification |
| Base model | Hailay/VEXMLM |
| Vocabulary | 280,002 |
| Labels | 11 |
| Seeds published | 42, 43, 44, 45, 46 |
Labels cover PER, ORG, LOC, DATE and MISC in BIO format (11 classes).
Five-seed benchmark evaluation
Fine-tuned independently under seeds 42–46 with one configuration (hash
ce27cc194946) on an A100-PCIE-40GB. Reported as mean ± standard deviation over
the five runs, on the dataset's test split.
| Metric | Score |
|---|---|
| Entity-F1 | 72.82 ± 0.79 |
| Macro-F1 | 82.19 ± 0.69 |
| Accuracy | 95.15 ± 0.05 |
These are the paper's verified results. They come from the five-seed evaluation described above — not from interactive use.
Interactive inference vs. benchmark
Benchmark evaluation is the five-seed measurement on the held-out test split, shown in the table above.
Interactive inference is what the usage example below performs: Enter arbitrary Tigrinya text and inspect the predicted entity spans. Predictions on arbitrary user input are demonstrations only and do not produce or reproduce the benchmark score.
Repository layout
Five independently fine-tuned checkpoints, one per seed. The reported benchmark score is the mean ± standard deviation over all five; no single seed is the "five-seed model."
seed-42/ seed-43/ seed-44/ seed-45/ seed-46/
Load a specific seed with the subfolder argument, as in the example below.
Fine-tuning
Fine-tuned from Hailay/VEXMLM, a
vocabulary-extended XLM-R (280,002 subwords, 30,000 Ge'ez tokens merged into the
SentencePiece model) after continued MLM pretraining.
| Hyperparameter | Value |
|---|---|
| Max sequence length | 256 |
| Batch size | 32 |
| Epochs | 4 |
| Learning rate | 2e-5 |
| LR schedule | Linear decay, 10% warmup |
| Weight decay | 0.01 |
| Gradient clipping | 1.0 |
| Optimizer | AdamW (β₁ 0.9, β₂ 0.999, ε 1e-8) |
| Precision | bf16 |
| Trainable parameters | All |
| Hardware | 1× NVIDIA A100 |
Runs are bit-reproducible: enable_full_determinism,
CUBLAS_WORKSPACE_CONFIG=:4096:8, dataloader_num_workers=0.
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
repo = "Hailay/VEXMLM-Tigrinya-NER"
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="seed-42")
model = AutoModelForTokenClassification.from_pretrained(repo, subfolder="seed-42")
model.eval()
words = "ኤርትራ ኣብ ቀርኒ አፍሪቃ እትርከብ ሃገር እያ።".split()
enc = tokenizer(words, is_split_into_words=True, return_tensors="pt", truncation=True)
with torch.no_grad():
pred = model(**enc).logits.argmax(-1)[0].tolist()
seen = set()
for p, w in zip(pred, enc.word_ids(0)):
if w is None or w in seen:
continue
seen.add(w)
print(words[w], "->", model.config.id2label[p])
Limitations
- Fine-tuned for Tigrinya on Tigrinya NER only; performance on other languages, domains or label schemes is not characterised.
- The base model covers Amharic and Tigrinya; other Ge'ez-script languages were not part of pretraining.
- The pretraining text is of undocumented origin (licence unknown); samples contain religious translations alongside general web prose, and the model may reflect those distributions and any biases present in them.
- Single-configuration study: no hyperparameter search was performed, and baseline comparisons in the paper are single-seed.
Reproducibility
The fine-tuning launcher, evaluation code and per-run result records are in the official repository: https://github.com/hailaykidu/VEXMLM
sbatch scripts/slurm_stage2_spm_seeds.sh # 6 tasks × 5 seeds
python3 evaluation/export_spm_results.py # regenerates the metrics table
Citation
@inproceedings{teklehaymanot2026vexmlm,
title = {Vocabulary Expansion for Low-Resource African Languages:
A Case Study in Amharic and Tigrinya},
author = {Teklehaymanot, Hailay Kidu and Yadeta, Debela Desalegn and
Nejdl, Wolfgang},
booktitle = {Proceedings of the Workshop on Language Models for
Underserved Communities (LM4UC) at IJCAI},
year = {2026}
}
Accepted at the LM4UC Workshop, IJCAI 2026.
License
Apache 2.0, following xlm-roberta-base.