East Africa OmniASR 300M
This model grew out of my entry in the
AfriVoices East Africa ASR Hackathon.
As a Kalenjin developer who had previously worked on translation and TTS, I
used the challenge as the push to explore speech recognition for languages I
care about. I also wanted to try a strong CTC system instead of following the
usual Whisper route, so I built this project around Meta's official
omniASR_CTC_300M
model from the
Omnilingual ASR project.
The result is a 325M-parameter CTC model for Swahili, Kikuyu, Dholuo, Somali, Kalenjin, and Maasai. I continued improving it after the hackathon, with the goal of turning the competition checkpoint into a model that can be tested and used outside Kaggle.
Training data
The supervised mixture combines AfriVoices Kenya data for Kikuyu, Dholuo, Somali, Kalenjin, and Maasai with AfriVoices Swahili. It contains both scripted and natural unscripted speech, including regional accents, long-form speech, code-switching, and varied recording conditions.
The training sampler balanced the six languages at roughly 477–500 weighted hours each, approximately 2,900 sampler-weighted hours in total. Audio was standardized to mono 16 kHz and transcripts were normalized consistently. Known text-encoding inconsistencies were repaired before the final training run. The published checkpoint is from step 10,000.
The repository contains the step-10,000 fairseq2 checkpoint, FP32 and dynamic INT8 ONNX exports, the v1 tokenizer, and language-specific KenLM binaries.
Results
Development benchmark: 2,354 held-out AfriVoices clips across all six
languages. KenLM uses beam 64 and the checked-in decoder_config.json
parameters.
| Evaluation | Greedy WER | Greedy CER | KenLM WER | KenLM CER |
|---|---|---|---|---|
| Swahili | 16.00% | 3.96% | 9.23% | 2.90% |
| Kikuyu | 42.66% | 8.78% | 34.52% | 8.90% |
| Dholuo | 47.26% | 9.92% | 37.28% | 8.98% |
| Somali | 55.03% | 21.11% | 39.97% | 20.40% |
| Kalenjin | 70.81% | 17.28% | 61.43% | 19.68% |
| Maasai | 65.89% | 19.01% | 56.11% | 18.30% |
| Corpus/micro | 53.06% | 14.91% | 42.60% | 14.87% |
| Language macro | 49.61% | 13.34% | 39.76% | 13.19% |
The final decoder also obtained 38.956% WER on the hackathon's hidden late-submission leaderboard. That number is reported separately because the hidden set has no public per-language or CER breakdown.
WER and CER are error rates: lower is better. Decoder parameters were selected on development data, so the development KenLM scores are validation results, not an untouched test estimate.
CPU usage (greedy CTC)
from pathlib import Path
import numpy as np
import sherpa_onnx
import soundfile as sf
from huggingface_hub import snapshot_download
root = Path(snapshot_download(
"mutaician/east-africa-omniasr-300m",
allow_patterns=["onnx/model.int8.onnx", "onnx/tokens.txt"],
))
recognizer = sherpa_onnx.OfflineRecognizer.from_omnilingual_asr_ctc(
model=str(root / "onnx/model.int8.onnx"),
tokens=str(root / "onnx/tokens.txt"),
num_threads=2,
provider="cpu",
)
audio, sample_rate = sf.read("audio.wav", dtype="float32")
if audio.ndim == 2:
audio = audio.mean(axis=1)
stream = recognizer.create_stream()
stream.accept_waveform(sample_rate, np.asarray(audio, dtype=np.float32))
recognizer.decode_stream(stream)
print(stream.result.text)
Use mono 16 kHz audio for reproducible behavior. The KenLM binaries and exact
alpha/beta values are under kenlm/ and decoder_config.json; the companion
Hugging Face Space will expose greedy, balanced, and full beam-64 modes.
Repository layout
onnx/model.int8.onnx: recommended free-CPU deployment model.onnx/model.onnx: FP32 reference export.onnx/tokens.txt: complete CTC vocabulary.fairseq2/checkpoint/model/: original step-10,000 checkpoint shards.fairseq2/config.yaml: training configuration.tokenizer/omniASR_tokenizer.model: OmniASR v1 tokenizer.kenlm/: six binary 5-gram language models and vocabularies.normalization.json: audio, chunking, and text normalization contract.decoder_config.json: language routing and final KenLM parameters.evaluation.json: aggregate and per-language WER/CER metrics.
Limitations
Kalenjin and Maasai remain the weakest languages in the development benchmark. KenLM improves WER substantially but can slightly worsen CER for some languages. The model does not perform automatic language identification; pass the language explicitly when using language-model decoding. Accuracy on new recording conditions can differ from these development and leaderboard scores.
Attribution
Fine-tuned from Meta's official
omniASR_CTC_300M
model and v1 tokenizer from
Omnilingual ASR, released
under Apache 2.0. Training speech comes from the AfriVoices East Africa
hackathon data; see the
AfriVoices-KE paper for more information on
the Kenyan-language collection.