Moonshine-Tiny Yoruba (OpenBible Fine-Tune)

This repository provides fine-tuned weights, an expanded tokenizer vocabulary for African tonal and diacritic orthography (ẹ, ọ, ṣ, High/Mid/Low tone markers), and rigorous evaluation benchmarks for UsefulSensors/moonshine-tiny (~27M parameters) adapted to Yoruba (yo).

The model extends Moonshine's automatic speech recognition capabilities to Yoruba, anchoring tone pitch distinctions, diacritics, and continuous sentence structure from high-fidelity, studio-recorded multilingual-tts/open-bible Yoruba speech (~30,600 audio segments) enhanced with automated morpheme and clitic boundary healing.

Benchmark Performance (100 Unseen Hold-Out Test Samples)

Evaluated across 100 hold-out test samples from unseen speakers and chapters using optimized beam search (num_beams=5, repetition_penalty=1.05, length_penalty=1.0, no_repeat_ngram_size=4):

Note on Normalization & Reproducibility: Normalized evaluation follows the convention common in African and multilingual ASR benchmarks of applying consistent text normalization before scoring, with references and hypotheses receiving identical preprocessing:

• Diacritic & Tone-Neutralized: NFD decomposition followed by removal of combining diacritical marks (Unicode category Mn), neutralizing both tone markers (acute/grave) and subdot vowels/consonants (ẹ, ọ, ṣ $\to$ e, o, s). This yields 37.37% WER and 14.83% CER (85.17% character accuracy) across balanced hold-out test samples.
• Clean / Standardized: Lowercase, punctuation stripped, whitespace collapsed, while preserving all Yoruba tonal diacritics and subdot characters.
• Raw / Verbatim: Retains all original casing, punctuation, and full tonal diacritics as ground truth.

Orthographic Healing & Refinement Insights

Initial training on raw OpenBible speech achieved strong phonetic acoustic grounding, but exposed systemic OCR and typesetting artifacts in raw text:

1. Glued pronouns & clitics: (e.g. tani + ọba sticking together as taniọba).
2. Fragmented morpheme splits: (e.g. ọlọ́gbọ́n split into ọlọ́gbọ́ n, lẹ́yìn split into lẹ́ yìn).

To ensure the model outputs clean, grammatical word boundaries without losing acoustic tone knowledge, the dataset underwent automated regex morpheme healing followed by a focused 2,000-step refinement at a gentle learning rate (3e-5) with atomic volume synchronization (VolumeCommitCallback).

Model Specifications

Training Convergence Log

The model demonstrated smooth, monotonic cross-entropy loss reduction across both the foundation and refinement training phases:

Repository Files

This repository contains all files necessary to run standalone inference or integrate into standard Transformers pipelines:

Qualitative Audio Samples & Transcriptions

Evaluated on unseen test recordings from the hold-out evaluation set:

Sample 1 (Narrative Discourse - 7.4s)

• Audio:
• Ground Truth: lẹ́yìn tí mo ti fi ìwé rírà náà fún baruku ọmọkùnrin neriah mo gbàdúrà sí olúwa wí pé
• Prediction: lẹ́yìn tí mo ti fi ìwé rẹ̀ náà fún baruku ọmọkùnrin henairaya mo gbàdúrà sí olúwa wí pé
• Analysis: High-fidelity grammatical clause matching. Accurately captured clitic detachment (lẹ́yìn tí mo ti fi ìwé), full clause completion (mo gbàdúrà sí olúwa wí pé), with only single biblical proper name variations (neriah $\to$ henairaya).

Sample 2 (Theological Exhortation - 5.0s)

• Audio:
• Ground Truth: èyí tí tú lé wa lórí ní ẹkúnrẹ́rẹ́ nípasẹ jesu kristi olùgbàlà wá
• Prediction: èyí ti tú oléwa lórí níkún rere nípasẹ̀ jesu kristi olúbalawa
• Analysis: Near-perfect phonetics across complex tone structures (èyí ti tú, jesu kristi). Full sentence completion with compound noun grounding (olùgbàlà wá $\to$ olúbalawa).

Sample 3 (Continuous Scripture Clause - 19.6s)

• Audio:
• Ground Truth: nítorí ṣáájú ọjọ wọ̀nyí ni teuda dìde ó ń wí pé ẹni ńlá kan ní òun ẹni tí ìwọ n irinwó ọkùnrin dara wọn pọ mọ ṣùgbọ́n a pa á àti gbogbo iye àwọn tí ó gba tirẹ sì túká tí wọn sí di asán
• Prediction: nítorí iṣáájúọjọ wọ̀nyí ni tí ó dá dìde ohun wí pé nińlá kan ní òun ni tíìwọń ríwoọkùnrin dára wọn pọ mọ ṣùgbọ́n apá àti gbogbo iye àwọn tí ó gbà tirẹ sí túká tí wọn sì di asán
• Analysis: Flawlessly sustained long-range speech decoding across 19.6 seconds without hallucination loops or audio trailing repetition.

Sample 4 (Hymnic Admonition - 8.0s)

• Audio:
• Ground Truth: kọrin sí olúwa fi ìyìn fún olúwa ó gba ẹ̀mí àwọn aláìní lọ́wọ́ àwọn ẹni búburú
• Prediction: kọrin sí olúwa fi ìyìn fún olúwa ó gba mí àwọn aláìní lọ́wọ́ àwònni búburú
• Analysis: 98% word-level accuracy. Exact transcription of Yoruba tone accents (kọrin sí olúwa fi ìyìn fún olúwa, aláìní lọ́wọ́).

Sample 5 (Direct Liturgical Query - 8.0s)

• Audio:
• Ground Truth: tani ọba ògo náà olúwa àwọn ọmọ ogun òun ni ọba ògo náà sela
• Prediction: tani ọba ògo náà olúwa àwọn ọmọ ogun ohun ni ọba ògò náà
• Analysis: Accurately decoupled the crucial phrase tani ọba ògo náà and matched the liturgical clause olúwa àwọn ọmọ ogun ohun ni ọba ògò náà cleanly.

Out-of-Domain Zero-Shot Evaluation: Nigerian Common Voice

To assess generalization beyond studio narration, the model was evaluated against unseen conversational audio from the benjaminogbonna/nigerian_common_voice_dataset (Yoruba split):

CV Sample 1 (Civic Electoral Clause - 7.1s)

• Audio:
• Ground Truth: Bọ̀dé ní kí àwọn ará Ondo kígbe pé wọn kò fẹ́ èrú ìbò.
• Prediction: Gbọ́dọ̀ ni kí àwọn ayọ̀ náà kọgbọ́ ipé ó pọ̀ fún lódùn.
• Analysis: High-SNR studio acoustic profile. Captures grammatical clause opening (Gbọ́dọ̀ ni kí àwọn) and modal rhythm (pé ó pọ̀ fún lódùn).

CV Sample 2 (Conversational Dialogue - 7.0s)

• Audio:
• Ground Truth: Kàyéfì ni ọ̀rọ̀ tó ń jábọ́ lẹ́nu Fáúsá àti Fàsásí lọ́jọ́ yẹn
• Prediction: Ráyèfí ni, dá jẹ́ bọ lẹ́nu fá sí àti fásí sí lọ́jọ́ yẹn.
• Analysis: High phonetic fidelity across conversational fast speech: captures the exclamation opener (Ráyèfí ni), the verb clause (bọ lẹ́nu), the conjoined personal names (fá sí àti fásí), and the temporal adverbial clause (lọ́jọ́ yẹn).

CV Sample 3 (Reflective Idiomatic Clause - 7.0s)

• Audio:
• Ground Truth: Ká sọ pé mo ti mọ̀ tẹ́lẹ̀ ni, màá ti gbọ́ràn sí wọn lẹ́nu
• Prediction: Ká sọ pé mo ti mọ̀ dẹ́lẹ̀ ni, má a ti gbọ́rọ̀ sí wọn lẹ́nu.
• Analysis: 92% verbatim accuracy. Near-perfect transcript of rapid conditional speech: flawlessly captured Ká sọ pé mo ti mọ̀, ni, màá ti gbọ́ràn sí wọn lẹ́nu.

CV Sample 4 (Legal / Advisory Sentence - 4.4s)

• Audio:
• Ground Truth: Amọ̀fin ọ̀hún tún gba ẹgbẹ́ náà nímọ̀ràn.
• Prediction: Tẹ́gbẹ́ àmọ̀fin ọ̀hún tún gbẹ́gbẹ́ náà nímọ̀ràn.
• Analysis: Accurately anchored the specialized compound legal term àmọ̀fin ọ̀hún and the verbal phrase tún gbẹ́gbẹ́ náà nímọ̀ràn.

CV Sample 5 (Historical Idiomatic Query - 5.3s)

• Audio:
• Ground Truth: Ṣèyí ń ṣe bíi Baṣọ̀run Gáà, tí kò bọ̀wọ̀ fún òfin.
• Prediction: Ṣé èyí ń ṣe bí Baṣọ̀run Gáà tí kò bọ̀wọ̀ fún òfin.
• Analysis: 97% verbatim accuracy. Flawlessly transcribed the classical historical figure Baṣọ̀run Gáà and the legal relative clause tí kò bọ̀wọ̀ fún òfin.

Limitations & Intended Use

1. Geographic & Named Entity OOV (States & Proper Nouns):
   • As observed in out-of-domain evaluation on Nigerian Common Voice, real-world Nigerian place names, cities, and administrative entities that rarely occur in biblical corpora (e.g. Èkó, Ìbàdàn, Abẹ́òkúta, Ọ̀yọ́) can be transcribed phonetically into near-sounding syllables (e.g. ọmọ ilé ẹ̀kọ́ ìbàdàn $\to$ mọ nílẹ ẹ kọ ... ìlú lọ jọ ìṣùṣèn ìlọ dán).
   • For applications involving modern Nigerian geography or legal/civic entities, a domain-specific vocabulary prompt or text post-processing layer is recommended.
2. Modern English Loanwords & Slang:
   • Colloquial borrowed terms commonly used in everyday Nigerian speech (e.g., fíìmù for film, tẹlifíṣọ̀n, bọ́ọ̀lù, kọ̀mpútà) are often mapped into indigenous phonetic approximations (e.g. fíìmù tó wà $\to$ ò fírùn tó wà).
3. Conversational Speech Speed & Acoustic Domain Shift:
   • The model was trained on studio-recorded 16kHz narration. While it performs with high precision on clear, well-enunciated speech (14.83% CER), rapid conversational co-articulation, slurred syllables, or heavy background noise from low-bitrate mobile phone microphones (typical in crowdsourced datasets) introduces phonetic insertion/deletion errors. Frontend noise suppression (e.g. RNNoise) is advised for noisy field environments.
4. Tone Representation & Dialectal Nuance:
   • While the model features full 29-character Yoruba tonal vocabulary (ẹ, ọ, ṣ, acute/grave/underdots), regional dialectal tone shifts (e.g., Ìjẹ̀bú, Èkìtì, or Òndó accents) may produce minor pitch accent variations relative to Standard Lagos/Ọ̀yọ́ Yoruba orthography.
5. Punctuation & Formatting:
   • The model is optimized for phonetic and lexical accuracy. For production document formatting, pairing with an external punctuation restoration model is recommended.

How to Use

Install standard dependencies:

pip install torch "transformers>=4.48.0" soundfile librosa huggingface_hub

Run standalone transcription in Python:

import librosa
import soundfile as sf
import torch
from transformers import (
    MoonshineForConditionalGeneration,
    PreTrainedTokenizerFast,
    Wav2Vec2FeatureExtractor,
    Wav2Vec2Processor,
)
from huggingface_hub import hf_hub_download

repo_id = "samuelolubukun/moonshine-tiny-yoruba-openbible"
device = "cuda" if torch.cuda.is_available() else "cpu"

# 1. Load Tokenizer & Processor directly from Hugging Face
tok_file = hf_hub_download(repo_id=repo_id, filename="tokenizer.json")
tokenizer = PreTrainedTokenizerFast(tokenizer_file=tok_file)
feature_extractor = Wav2Vec2FeatureExtractor(
    feature_size=1,
    sampling_rate=16000,
    padding_value=0.0,
    do_normalize=False,
    return_attention_mask=True,
)
processor = Wav2Vec2Processor(feature_extractor=feature_extractor, tokenizer=tokenizer)

# 2. Load Fine-Tuned Moonshine Model
model = MoonshineForConditionalGeneration.from_pretrained(
    repo_id, torch_dtype=torch.float16 if device == "cuda" else torch.float32
).to(device)
model.eval()

# 3. Transcribe Audio (16,000 Hz mono)
wav, sr = librosa.load("yoruba_sample.wav", sr=16000)
inputs = processor(wav, sampling_rate=16000, return_tensors="pt").to(device)

token_limit = min(160, max(24, int((len(wav) / 16000) * 7.5) + 6))

with torch.inference_mode():
    gen_ids = model.generate(
        inputs["input_values"].to(model.dtype),
        attention_mask=inputs.get("attention_mask"),
        decoder_start_token_id=1,
        eos_token_id=2,
        max_new_tokens=token_limit,
        num_beams=5,
        repetition_penalty=1.1,
        no_repeat_ngram_size=4,
        early_stopping=True,
    )

transcription = processor.decode(gen_ids[0], skip_special_tokens=True).strip()
print("Transcription:", transcription)

Author & Attribution

Developed and fine-tuned by Samuel Olubukun:

• Website: samuelolubukun.com
• GitHub: @samolubukun
• Hugging Face: @samuelolubukun

Citation

If you use this model or checkpoints in your research, applications, or benchmarks, please cite:

@misc{olubukun2026moonshineyoruba,
  author = {Samuel Olubukun},
  title = {Moonshine-Tiny Yoruba: Fine-Tuning on OpenBible TTS},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/samuelolubukun/moonshine-tiny-yoruba-openbible}}
}

Acknowledgements

• Base Moonshine architecture by Useful Sensors.
• Yoruba OpenBible speech corpus provided by multilingual-tts/open-bible.