ACI Yoruba Phase X
Fine-tuned Wav2Vec2 acoustic model (shared with Phase 10) paired with a 5-gram KenLM language model (alpha=2.0, beta=1.5) trained on a larger unigram corpus, plus a phonetic post-processing map. Self-contained: model weights, KenLM decoder, and phonetic map are all bundled in this repo.
Evaluation
Measured on a held-out set of 55 samples -- specifically, the only samples in our AR_Audios dataset that were not used to fine-tune the shared acoustic weights (Phase X and Phase 10 use the same acoustic model; evaluating on any of the other ~495 samples would be testing on data the model was trained on, which does not reflect real-world performance).
Text normalization strips all punctuation symmetrically from both prediction and reference before scoring (the model's vocabulary does not produce punctuation, so scoring against punctuated references without this step misrepresents accuracy).
| Model | Decoder config | WER (held-out, normalized) |
|---|---|---|
| Phase 8 / 9 | KenLM alpha=0.5, beta=1.0 | 12.63% |
| Phase 10 | KenLM alpha=0.5, beta=1.0 (fine-tuned acoustics) | 12.33% |
| Phase X | KenLM alpha=2.0, beta=1.5, larger unigram corpus | 7.09% |
The stronger language-model weighting and larger unigram vocabulary account for the improvement over Phase 10 -- both share the same acoustic weights, so this gain comes specifically from decoding configuration, not additional acoustic training.
Caveat: 55 samples is a real but modest held-out set from a single dataset (AR_Audios). We recommend validating against your own audio, and ideally a larger or independent test set, before treating this as a fully generalized production benchmark.
Quickstart
pip install torch transformers pyctcdecode librosa
import re
import json
import torch
import librosa
from huggingface_hub import hf_hub_download
from transformers import Wav2Vec2ForCTC, Wav2Vec2ProcessorWithLM
REPO_ID = "Awesome-x/aci-yoruba-phaseX"
model = Wav2Vec2ForCTC.from_pretrained(REPO_ID)
processor = Wav2Vec2ProcessorWithLM.from_pretrained(REPO_ID)
map_path = hf_hub_download(repo_id=REPO_ID, filename="phonetic_map.json")
with open(map_path, encoding="utf-8") as f:
PHONETIC_MAP = json.load(f)
def clean_transcript(text):
for pattern, replacement in PHONETIC_MAP.items():
text = re.sub(pattern, replacement, text)
text = re.sub(r"[^\w\s]", "", text)
return re.sub(r"\s+", " ", text).strip().upper()
audio, sr = librosa.load("sample.wav", sr=16000)
input_values = processor(audio, sampling_rate=16000, return_tensors="pt").input_values
with torch.no_grad():
logits = model(input_values).logits
raw_transcript = processor.decode(logits[0].numpy()).text
print("Transcript:", clean_transcript(raw_transcript))
Status
Production candidate -- best verified performance among current checkpoints on the held-out benchmark above.