whisper-small-lingala-qlora-merged

openai/whisper-small adapted to Lingala through QLoRA/LoRA fine-tuning, with the adapters merged into the base weights. It converts spoken Lingala into text. Developed by Congo Digital Services (CDS SARL) for the MALOBA project, with UNDP Republic of Congo.

Sister repository, with the LoRA adapters only: Congo-digital-service/whisper-small-lingala-qlora-adapters

Summary

Uses

Direct use: transcription of Lingala speech, either with transformers or through a compatible inference server exposing an OpenAI-style transcription endpoint (see Deployment specification).

Out of scope:

• production deployment without review, since the model and its endpoint are not yet validated;
• high-stakes transcription (legal, medical, safety-critical audio) without human review;
• other languages;
• speaker identification.

How to use

import librosa
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

model_id = "Congo-digital-service/whisper-small-lingala-qlora-merged"
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id).to(device).eval()

audio, _ = librosa.load("audio.wav", sr=16000)  # Whisper expects 16 kHz mono
features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)
with torch.no_grad():
    ids = model.generate(input_features=features, language="ln", task="transcribe")
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Always resample to 16 kHz, and pass language="ln" and task="transcribe". Without the language token, Whisper guesses the language and may transcribe or translate into another one.

Training data

• Dataset: Congo-digital-service/audios-lingala-annotatees-v2, train split. It holds annotated Lingala speech from Radio Rurale broadcasts and the Voice-a-thon campaign, segmented and transcribed by MALOBA annotators.
• Spelling: transcriptions follow the annotators' spelling. It is not yet orthographically harmonised: the same word can appear in several spellings, and the Lingala-specific letters ɔ and ɛ are rarely written.
• Known data issue: 17 test segments have audio identical to segments in the training split (see Evaluation).

Training procedure

• QLoRA on openai/whisper-small: 4-bit NF4 quantisation of the base model and trained LoRA adapters.
• 2,500 steps, with evaluation every 250 steps. The checkpoint with the best validation WER was retained, then the adapters were merged and the merged model exported.
• Framework: PEFT 0.20.0, transformers.

Training-time validation curve (original training notebook, its own protocol). These values are not the reference scores, which are given under Evaluation.

Validation loss decreases steadily, with no clear sign of overfitting.

Evaluation

Protocol (run 2026-09-16T11-18-09Z):

• Test set: audios-lingala-annotatees-v2 @ 57c0c603, split test, 1,882 segments from 20 source recordings.
• Aggregation: micro, over the whole corpus.
• Two forms reported:
  • raw: text as is;
  • normalised: Unicode NFC, with ɔ/ɛ/ε neutralised.

These scores were measured with the adapters (revision 501d57ff071ee7a871a093a5a15a62088da1ea30), which are the weights merged into this model.

• Fine-tuning effect: WER divided by 4.2 and CER by 5.3 against the base model. The base model does not know Lingala: a WER above 100 % means it outputs more wrong words than the reference contains.
• Architecture comparison: MMS, another architecture allowed by the project's terms of reference, reaches 60.1 % WER without any MALOBA data. XLS-R cannot be measured without fine-tuning, since it has no recognition head.
• Human reference floor: two annotators transcribing the same audio disagree by 12.1 % CER. This is the practical lower bound of the error rate on this data.
• External reference: Nahabwe et al. (2025) report 19.85 % WER for fine-tuned XLS-R on about 50 h of Lingala. That result uses a different test set, so it is a reference point, not a direct comparison.

Qualitative example. For the reference "bon nalonaka bandunda lokola épinard":

• the base model outputs an unrelated French sentence;
• this model outputs "Bonalonaka bandunda lokola epinard".

Human evaluation (3 validators, 25 outputs): 16 % judged correct by majority, 100 % recoverable, 0 % incorrect. Inter-rater agreement is too low for this verdict to be published as a rate (Krippendorff's α = 0.258, below the 0.60 threshold set before the campaign). The grading grid is being revised.

Superseded figure: the 53.38 % WER / 22.12 % CER previously published on this card are training-time validation values, measured under an earlier dataset version and protocol. They are not comparable to the scores above and are withdrawn as reference scores.

Deployment specification (API)

Set language=ln in API requests, for the same reason as in the code example.

Validation status

The merged model is available on Hugging Face. The application-level endpoint remains to be deployed and validated: it still needs a stable production URL, an operational authentication key, and real end-to-end audio inference tests through the LiteLLM routing and the final inference provider, with latency measurement.

Limitations

• High error rate: 46.4 % WER. Transcriptions are drafts that need human review, not publication-ready output. The project's internal target is below 15 % WER. That target is a CDS target, not a requirement of the terms of reference, and it predates the measurement of the 12.1 % human floor.
• Spelling variation in the training transcriptions accounts for a large share of word errors: validators classified 65 % of the errors they found as spelling errors. Harmonising the corpus spelling is expected to reduce WER without retraining the acoustic model.
• Test set overlap: 17 of the 1,882 test segments share audio with the training split. The estimated effect is about 0.14 WER points. A re-evaluation on the leak-free V3 test set is planned.
• Small effective sample: the test set comes from 20 source recordings only, so segment-level scores are correlated.
• Domain: mostly radio and field speech. Regional accents, dialectal variation and noisy conditions not represented in the training data are not evaluated.

Environmental impact

Training ran on a single GPU for 2,500 steps. The hardware type and energy use were not recorded, so emissions cannot be estimated precisely; see the Machine Learning Impact calculator (Lacoste et al., 2019).

Licence

• Model weights: MIT, the licence of the base model openai/whisper-small.
• Training data: the licensing of the MALOBA speech corpus is under legal review. The dataset card declares CC BY 4.0. The project intends to apply the Nwulite Obodo Open Data License (NOODL-1.0). Until this is resolved, CDS asks users from high-income countries or commercial entities to publicly credit the MALOBA Project (UNDP Republic of Congo — language digitalisation initiative) in any derived publication, product, model or output.

Citation

@misc{cds2026whisperlingala,
  title        = {whisper-small-lingala-qlora},
  author       = {{Congo Digital Services}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Congo-digital-service/whisper-small-lingala-qlora-merged}},
  note         = {MALOBA project, UNDP Republic of Congo}
}

@inproceedings{radford2023whisper,
  title     = {Robust Speech Recognition via Large-Scale Weak Supervision},
  author    = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  booktitle = {Proceedings of the 40th International Conference on Machine Learning},
  pages     = {28492--28518},
  year      = {2023}
}

Contact

• Organisation: Congo Digital Services (CDS SARL)
• Website: congo-digital.com
• Email: info@congo-digital.com

Card updated on 28 September 2026.