Code-Switching Language ID — Roman Urdu / English
Token classification model that labels each word in Roman Urdu text as URD (Roman Urdu),
ENG (English), or MIX (nativized English loanword used as Urdu vocabulary).
Fine-tuned from xlm-roberta-base.
Dataset
- Source: filtered from the Roman Urdu Data Set (Sharf, 2017, UCI ML Repository, CC BY 4.0)
- 200 sentences, 3182 word-level labels
- Label distribution: URD 4367, ENG 271, MIX 178
- Dataset card: https://huggingface.co/datasets/qandeelasim13/code-switching-codesaviours-si26-qandeel
Evaluation (held-out 20% split)
| Label | F1 | Precision | Recall | Support |
|---|---|---|---|---|
| URD | 0.958 | 0.983 | 0.935 | 859 |
| ENG | 0.683 | 0.609 | 0.778 | 54 |
| MIX | 0.606 | 0.476 | 0.833 | 36 |
Overall weighted F1: 0.929
Limitations
- MIX is severely underrepresented in training data (12 of 3182 word labels), so MIX predictions should be treated as low-confidence until the dataset is expanded.
- Trained on informal social-media Roman Urdu; may not generalize well to formal text.
Training
XLM-RoBERTa-base fine-tuned for 8 epochs with a class-weighted loss (weights: URD 0.37, ENG 5.92, MIX 9.02) to counter the class imbalance above.
Built for Code Saviours Summer Internship 2026 (SI-26), Week 7.