meutsabdahal/nepali-ner-roberta
A Nepali named-entity recognition model: IRIISNEPAL/RoBERTa_Nepali_110M fine-tuned on EverestNER. Published as one row of a six-encoder ablation comparing base models under an identical pipeline and protocol. See the benchmark repo for the full comparison, method, and error analysis.
Model description
- Base model: IRIISNEPAL/RoBERTa_Nepali_110M
- Task: token classification (NER), 5 entity types: Person, Location,
Organization, Date, Event (IOB2 scheme, 11 labels including
O) - Training data: EverestNER (~15.8k sentences of Nepali news text)
- Checkpoint: seed 13 of 3 trained seeds (13, 21, 42); this seed was selected as the one closest to the across-seed mean micro-F1, so it is representative rather than a best-of-N pick
Intended uses and limitations
Intended for named-entity recognition on Nepali text, particularly news-domain text similar to EverestNER's source material (political, geographic, and organizational names skew are expected to be well covered; other domains may see degraded performance).
EVENT is unreliable and should not be trusted as a ranking signal. The EverestNER test set has only 59 Event instances, so its F1 (see table below) carries a wide margin of error and swung between 0.25 and 0.61 across the six models compared in the parent benchmark. Do not build a claim on this model's Event performance specifically.
This model is a research/portfolio artifact, not a production service. It has not been evaluated outside the EverestNER domain.
Training procedure
Identical protocol across every model in the ablation (no per-model hyperparameter tuning, so comparisons between base models are not confounded by which one got a luckier learning rate):
| hyperparameter | value |
|---|---|
| learning rate | 5e-05 |
| epochs | 4 |
| batch size | 16 |
| weight decay | 0.01 |
| warmup ratio | 0.1 |
| max sequence length | 256 |
| checkpoint selection | best dev micro-F1 (seqeval, strict IOB2) |
| seeds | 13, 21, 42 |
Evaluation results
EverestNER test set (1,950 sentences), seqeval strict IOB2. Mean +- std across 3 seeds (optimization variance):
| metric | value |
|---|---|
| micro-F1 | 0.9185 +- 0.0020 |
| macro-F1 | 0.8557 +- 0.0024 |
Per-entity (mean across seeds):
| entity | precision | recall | f1 |
|---|---|---|---|
| PER | 0.957 | 0.938 | 0.947 |
| LOC | 0.916 | 0.906 | 0.911 |
| ORG | 0.895 | 0.904 | 0.900 |
| DATE | 0.944 | 0.923 | 0.933 |
| EVENT | 0.562 | 0.616 | 0.587 |
On this seed's own checkpoint, a sentence-level bootstrap over the test set gives a 95% CI of [0.9093, 0.9282] on micro-F1 (test-sampling variance; distinct from the seed mean/std above, which is optimization variance; the two are not combined).
How to use
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
tokenizer = AutoTokenizer.from_pretrained("meutsabdahal/nepali-ner-roberta")
model = AutoModelForTokenClassification.from_pretrained("meutsabdahal/nepali-ner-roberta")
ner = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="simple")
ner("काठमाडौं महानगरपालिकाले नयाँ योजना ल्यायो।")
License
EverestNER is released for non-commercial use only. This model is a derivative fine-tuned on that data and inherits the same restriction; it is not licensed for commercial use. The training/evaluation code for this project is separately licensed under Apache-2.0; that code license does not extend to this model's weights. See the dataset's own repository for the exact terms.
Citation
If you use this model, please credit EverestNER and the base encoder:
- EverestNER: https://github.com/nowalab/everest-ner
- Base model: IRIISNEPAL/RoBERTa_Nepali_110M