mms-300m-fongbe
This model is a fine-tuned version of facebook/mms-300m specifically for Fongbe (Fon), a tonal language primarily spoken in Benin.
It was developed to preserve linguistic integrity by maintaining critical tonal diacritics and unique orthographic characters (e.g., ɖ, ɛ, ɔ, è, é). This model achieves State-of-the-Art (SOTA) results for Fongbe Automatic Speech Recognition (ASR) on the ALFFA test benchmark.
📊 Evaluation Results
The model was evaluated on the held-out ALFFA test set (2,168 utterances):
| Metric | Score |
|---|---|
| WER (Word Error Rate) | 0.0948 (9.48%) |
| CER (Character Error Rate) | 0.0396 (3.96%) |
Benchmark Comparison (with diacritics)
| Model | WER (%) | CER (%) | Year |
|---|---|---|---|
| Laleye et al. (Baseline) | 44.04% | — | 2016 |
| MMS-300m-Fongbe (Ours) | 9.48% | 3.96% | 2026 |
Inference Examples
| Reference | Prediction | Result |
|---|---|---|
| gannu elɔ kpɔ hu ɖe ɔ | gannu elɔ kpɔ hu ɖe ɔ | ✅ Perfect |
| ɖɔla tεnwe | ɖɔla tεnwe | ✅ Perfect |
| ama e gbɔ mɔ ɖo nɔ ɔ nu e wε e nɔ ɖu | ama e gbɔ mɔ ɖo nɔ ɔ nu ɔ e nɔ ɖu | ⚠️ Minor error |
📖 Model Description
- Architecture: MMS (Massive Multilingual Speech) 300M parameter model.
- Methodology: Fine-tuned with Connectionist Temporal Classification (CTC) loss.
- Language: Fongbe (fon).
- Phonetic Representation: Tone-preserved orthography using NFD/NFC normalization.
- Special Features: Full support for Fon-specific characters (
ɖ,ɛ,ɔ) and tone markers.
🚀 How to Use
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="Professor/mms-300m-fongbe")
# Ensure your audio is 16kHz
transcription = asr("path_to_audio.wav")
print(transcription["text"])
🎯 Intended Uses & Limitations
Intended Uses
- High-accuracy transcription of Fongbe speech.
- Research in low-resource and tonal language modeling.
- Base model for downstream Fongbe NLP tasks (NLP4Fon).
Limitations
- Performance may degrade in noisy environments or with heavy background music.
- Primarily trained on continuous speech; may require further fine-tuning for specific dialects or extremely fast colloquial speech.
📁 Training and Evaluation Data
The model was trained on a consolidated dataset merging the ALFFA Project (African Languages in the Field) data and the Zenodo Fongbe Speech Dataset:
- Train + Validation Set: ~10.85 hours (Merged and re-split 90/10).
- Test Set: ~1.45 hours (Standard 2,168 utterances from ALFFA for benchmark consistency).
- Sampling Rate: 16,000 Hz.
⚙️ Training Procedure
Hyperparameters
- Learning Rate: 1e-4
- Effective Batch Size: 64 (Batch 16 x 4 Grad Accumulation)
- Optimizer: AdamW (Fused)
- Epochs: 30
- Precision: Mixed Precision (FP16)
- Hardware: NVIDIA H100 GPU
Training Logs
| Training Loss | Epoch | Step | Validation Loss | WER |
|---|---|---|---|---|
| 26.3861 | 3.11 | 500 | 1.0171 | 0.6021 |
| 2.5796 | 6.21 | 1000 | 0.3366 | 0.2600 |
| 1.3316 | 9.32 | 1500 | 0.2312 | 0.1799 |
| 0.9087 | 12.42 | 2000 | 0.2031 | 0.1557 |
| 0.6678 | 15.53 | 2500 | 0.1752 | 0.1397 |
| 0.5069 | 18.64 | 3000 | 0.1747 | 0.1325 |
| 0.4034 | 21.74 | 3500 | 0.1583 | 0.1137 |
| 0.3142 | 24.85 | 4000 | 0.1618 | 0.1147 |
| 0.2622 | 27.95 | 4500 | 0.1656 | 0.1085 |
📜 Citation & Credits
If you use this model in your research, please cite the following:
Dataset Contributors: Laleye, Fréjus A. A., et al. (ALFFA Project & Zenodo release).
Model Developer: Victor Olufemi (Professor).
@dataset{laleye_frejus_2022_6604637,
author = {Laleye, Fréjus A. A.},
title = {Fongbe Speech Dataset},
year = 2022,
publisher = {Zenodo},
doi = {10.5281/zenodo.6604637}
}
@inproceedings{laleye2016FongbeASR,
title={First Automatic Fongbe Continuous Speech Recognition System},
author={A. A Laleye, Fréjus and Besacier, Laurent and Ezin, Eugène C. and Motamed, Cina},
year={2016},
organization={FedCSIS}
}