whisper-uzbek-medium-callcenter
A Whisper-medium model fine-tuned for Uzbek speech-to-text, with data augmentation aimed at telephone / call-center audio (narrowband, noise, reverb) and anti-hallucination training on non-speech segments.
Training data
Fine-tuned on the ~780-hour Uzbek STT dataset:
Abduqayum/Uzbek-STT-Dataset-780h
(audiobooks, podcasts and tech talks; transcriptions generated with Google
Gemini). Base model: openai/whisper-medium (via an Uzbek-adapted checkpoint).
Training highlights:
- ~2 epochs, bf16, effective batch 16, LR 1e-5.
- Call-center augmentation on ~50% of clips: telephone narrowband (8 kHz codec + 300–3400 Hz band), background noise, light reverb, gain.
- Anti-hallucination: ~3% synthetic non-speech clips with empty labels, so the model stays silent on silence/noise instead of generating text.
Evaluation (Word Error Rate)
Numbers are normalized before scoring: Uzbek references write numbers as digits
(2010-yilda) while the model speaks them as words (ikki ming o'ninchi yilda),
so digits are converted to Uzbek words and apostrophes/punctuation/case are
unified for a fair WER.
| Dataset | Clips (test) | WER |
|---|---|---|
| FeruzaSpeech | 899 | 7.88% |
| Common Voice 17.0 uz | 12,348 | 13.37% |
| FLEURS uz | 4,165 | 14.40% |
Usage
from transformers import pipeline
asr = pipeline("automatic-speech-recognition",
model="Abduqayum/whisper-uzbek-medium-callcenter", device=0)
text = asr("audio.wav",
generate_kwargs={"language": "uz", "task": "transcribe"})["text"]
print(text)
faster-whisper (with anti-hallucination decoding)
Convert with ct2-transformers-converter, then:
from faster_whisper import WhisperModel
m = WhisperModel("path/to/ct2-model", device="cuda", compute_type="float16")
segments, _ = m.transcribe(
"audio.wav", language="uz",
vad_filter=True, condition_on_previous_text=False,
no_speech_threshold=0.6, temperature=0,
)
print(" ".join(s.text for s in segments).strip())
For stereo call recordings, split the left/right channels and transcribe each separately.
Limitations
- Training transcriptions are model-generated (Gemini), so a small amount of label noise is expected.
- Tuned for Uzbek Latin script; Cyrillic output is not targeted.