Yoruba Politeness Classifier
Project Overview
This project develops a text classification model for identifying politeness levels in Yoruba, an African language that remains comparatively underrepresented in NLP resources.
The model classifies Yoruba utterances into three categories:
- NEUTRAL
- POLITE
- IMPOLITE
Dataset
The project began with an original Yoruba conversational dataset and was subsequently adapted and prepared for model training.
The final working dataset contained 541 annotated Yoruba utterances.
Class distribution:
- NEUTRAL: 259
- POLITE: 240
- IMPOLITE: 39
The data were divided into:
- Training set: 430 examples
- Test set: 108 examples
Method
The dataset was prepared through data adaptation, cleaning, annotation, label encoding and train-test splitting.
The fine-tuned model is based on XLM-RoBERTa and was trained for 5 epochs.
Training Results
Final training loss: 0.136959
Final validation loss: 0.399099
Overall reported training loss: 0.268594
Benchmark Results
Accuracy: 89.81%
Weighted Precision: 0.89
Weighted Recall: 0.90
Weighted F1-score: 0.89
Macro F1-score: 0.74
Per-class results
| Class | Precision | Recall | F1 |
|---|---|---|---|
| NEUTRAL | 0.88 | 0.94 | 0.91 |
| POLITE | 0.94 | 0.96 | 0.95 |
| IMPOLITE | 0.67 | 0.25 | 0.36 |
Research Observation
The model performed strongly on NEUTRAL and POLITE utterances. Performance on IMPOLITE utterances was considerably lower, reflecting the smaller representation of this class in the dataset.
This result highlights the importance of balanced and linguistically diverse resources for Yoruba NLP.
Intended Use
The model is intended for research and educational purposes in Yoruba natural language processing, computational linguistics and African language technology.
Limitations
The dataset is relatively small and the IMPOLITE category is underrepresented. Therefore, the reported overall accuracy should be interpreted together with the macro-averaged metrics and per-class results.