whisper-small-hu

openai/whisper-small fine-tuned for Hungarian speech recognition on Mozilla Common Voice 26.0.

Both models evaluated on the full Common Voice 26.0 Hungarian test split (13,127 clips) with the same pipeline: greedy decoding, forced <|hu|><|transcribe|> prompt, predictions and references lowercased and whitespace-normalized before scoring.

Usage

from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="SubZtep/whisper-small-hu")
print(asr("audio.mp3", generate_kwargs={"language": "hungarian", "task": "transcribe"})["text"])

Training

• Data: Common Voice 26.0 Hungarian train split (62,284 clips), silence-trimmed, 16 kHz
• Schedule: 3 epochs (~11,700 steps), effective batch size 16 (8 × 2 gradient accumulation)
• Optimizer: AdamW, lr 1e-5, linear decay with 500 warmup steps, fp16, gradient checkpointing
• Hardware: single NVIDIA T4 (Google Colab)
• Selection: best checkpoint by WER on a fixed 1,500-sample subset of the dev split

Training code: https://github.com/SubZtep/whisper-hu

Training progress

WER on the 1,500-sample dev subset during training:

Framework versions

• Transformers 4.44.2
• PyTorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.19.1

Limitations

• Trained on read speech (Common Voice sentences) — expect higher error rates on spontaneous conversation, noisy audio, or domain-specific vocabulary
• WER is measured on lowercased text; the model itself outputs casing and punctuation, but their accuracy is not reflected in the score
• Hungarian is agglutinative, so WER punishes single-suffix mistakes as whole-word errors — the CER of 3.33 is a better sense of how close transcripts are