muril-ner-multilingual 🏷️
This model is a fine-tuned version of google/muril-base-cased for Named Entity Recognition (NER) across three major Indic languages: Marathi (mr), Bengali (bn), and Telugu (te).
It uses a joint multilingual full fine-tuning strategy to extract three primary entity types:
- 👤 PER (Person)
- 🏢 ORG (Organization)
- 📍 LOC (Location)
Model Description
- Developed by: Learner-sai
- Model Type: Token Classification (NER)
- Base Model:
google/muril-base-cased - Languages: Marathi (
mr), Bengali (bn), Telugu (te) - License: Apache 2.0
MuRIL (Multilingual Representations for Indian Languages) was adapted by adding a 7-class linear sequence classification head (O, B-PER, I-PER, B-ORG, I-ORG, B-LOC, I-LOC). All parameters were updated during joint training across all three target languages to leverage cross-lingual transfer.
Intended Uses & Limitations
Intended Uses 🎯
- Automated entity extraction from news articles, social media, and documents in Marathi, Bengali, and Telugu.
- Downstream NLP applications such as information retrieval, question answering, and knowledge graph construction for Indic languages.
Limitations ⚠️
- Grammatical Suffixes: In agglutinative or highly inflected languages like Marathi and Telugu, locative or case suffixes (e.g., Marathi
-तin "दिल्लीत") may sometimes be included inside the predicted entity span. - Entity Scope: The model is trained exclusively on
PER,ORG, andLOCtags; it will not recognize other categories like dates, monetary values, or product names.
Training and Evaluation Data
The model was trained on a combined dataset comprising annotated sentences across Marathi, Bengali, and Telugu.
- Label Schema: IOB2 format (
B-,I-,O) with 7 total classes. - Token Alignment: Subword tokenization artifacts were handled using
-100label masking on non-initial subwords to ensure clean cross-entropy loss calculation.
Training Procedure
Training Hyperparameters
- Learning Rate:
3e-05 - Train Batch Size: 16
- Eval Batch Size: 32
- Epochs: 3
- Optimizer:
AdamW (fused)with $\beta_1=0.9, \beta_2=0.999, \epsilon=1\text{e-}08$ - LR Scheduler: Linear with 937 warmup steps (~10% of total steps)
- Mixed Precision: Native FP16 (
fp16=True) - Seed: 42
Evaluation Metrics
Evaluated on the validation split using seqeval (entity-level span matching):
| Epoch | Training Loss | Validation Loss | Precision | Recall | Entity F1 🏆 | Token Accuracy |
|---|---|---|---|---|---|---|
| 1.0 | 0.2845 | 0.2822 | 0.7172 | 0.7725 | 0.7438 | 0.9320 |
| 2.0 | 0.2024 | 0.2335 | 0.7392 | 0.7723 | 0.7554 | 0.9346 |
| 3.0 | 0.1804 | 0.2328 | 0.7382 | 0.7771 | 0.7572 | 0.9352 |
Framework Versions
- Transformers: 5.14.1
- PyTorch: 2.11.0+cu128
- Datasets: 2.21.0
- Tokenizers: 0.22.2