DistilBERT Telugu NER
Fine-tuned distilbert-base-multilingual-cased for Named Entity Recognition on Telugu, trained on the Naamapadam benchmark (Mhaske et al., ACL 2023).
Research context
Telugu (~80M speakers) is a low-resource language for NLP. Existing SOTA Telugu NER models like IndicNER achieve ~0.80 F1 but rely on 180M+ parameter models pretrained specifically on Indian languages. This work explores whether a smaller, distilled multilingual model (66M params, 40% smaller than BERT) can deliver acceptable NER performance via transfer learning — addressing the accuracy-vs-efficiency tradeoff that decides real-world deployability.
Results
Entity-level F1 on the manually annotated Naamapadam Telugu test set (847 sentences):
| Metric | Score |
|---|---|
| F1 | 0.7803 |
| Precision | 0.7827 |
| Recall | 0.7779 |
| Accuracy | 0.9389 |
Trained on 50k sentences (10%) of Naamapadam Telugu with fp16 mixed precision on a single Colab T4 GPU. Total training time: ~11 min.
Model details
- Model size: 66M parameters (40% smaller than BERT-base)
- Base:
distilbert-base-multilingual-cased - Task: Token classification (7-label BIO scheme)
- Labels:
O,B-PER,I-PER,B-ORG,I-ORG,B-LOC,I-LOC
Usage
from transformers import pipeline
ner = pipeline(
"token-classification",
model="chandanaau/distilbert-telugu-ner",
aggregation_strategy="first",
)
text = "నరేంద్ర మోదీ న్యూ ఢిల్లీలో మాట్లాడారు"
print(ner(text))
Known limitations
- Domain sensitivity: Naamapadam is auto-projected from English news, so tech and space-industry entities (Infosys, ISRO, Skyroot) are underrepresented and often missed.
- Training data volume: 10% of the full corpus used — full-data training likely reaches ~0.80 F1.
Live demo
Streamlit app (link coming after deployment)
Citation
If you use this model, please also cite the underlying dataset:
@inproceedings{mhaske-etal-2023-naamapadam,
title = "Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages",
author = "Mhaske, Arnav and Kedia, Harshit and Doddapaneni, Sumanth and others",
booktitle = "ACL 2023",
year = "2023"
}
About
Trained by Chandana Nuthangi. Full training code, evaluation scripts, and a live Streamlit app are available at the project repository: chandana-nuthangi/telugu-ner-thesis.
License: MIT