Code-Switching Language ID Model — Code Saviours SI-26 (Amna)
Model Description
This model fine-tunes xlm-roberta-base for
token classification on Roman Urdu–English code-switched text. Given a sentence like
"Aaj mera mood nahi hai for anything", it labels every word as:
- URD — Roman Urdu
- ENG — English
- MIX — a hybrid token (Urdu morphology applied to an English root, e.g.
tension-wention)
Training Data
Fine-tuned on the
code-switching-codesaviours-si26-amna
dataset — 160 real, naturally occurring Roman Urdu-English sentences (1,677 labelled words)
collected from Pakistani Twitter/X, Reddit (r/pakistan), YouTube comments, and everyday
WhatsApp-style phrasing.
Training Procedure
- Base model:
xlm-roberta-base - Task: Token classification (3 labels: URD, ENG, MIX)
- Split: 128 training sentences / 32 test sentences (80/20)
- Epochs: 5
- Batch size: 16
Evaluation Results
Evaluated on the held-out test set (334 labelled word tokens):
| Label | Precision | Recall | F1-score | Support |
|---|---|---|---|---|
| URD | 0.990 | 0.981 | 0.985 | 206 |
| ENG | 0.946 | 0.984 | 0.965 | 125 |
| MIX | 0.000 | 0.000 | 0.000 | 3 |
Overall accuracy: 0.973 · Weighted F1: 0.969
Limitations
MIXscored 0 because the dataset only contains 11MIXexamples in total (a genuinely rare linguistic phenomenon — Urdu echo-reduplication on an English root), leaving too few training/test examples for the model to learn the pattern. MoreMIX-labelled data is needed to improve this.- Trained on a small (160-sentence), single-annotator dataset skewed toward casual, everyday topics — may not generalise well to formal or domain-specific text.
How to Use
from transformers import pipeline
classifier = pipeline(
"token-classification",
model="AmnaNoor123/code-switching-codesaviours-si26-amna",
aggregation_strategy=None,
)
result = classifier("Bhai kal ka plan finalize ho gaya, I'm super excited")
for token in result:
print(token['word'], '->', token['entity'])
Intended Use
Token-level language identification for Roman Urdu-English code-switched text — a building block for NLP tools (spell-checkers, sentiment models, chatbots) that need to understand how Pakistanis actually write online.
Citation
If you use this model, please credit: Code Saviours SI-26 Internship — Amna, 2026.