Whisper Tiny: fine-tuned for Indian English
Fine-tuned openai/whisper-tiny (39M params) on TIE_shorts, a corpus of NPTEL-style Indian-English academic lecture audio. Part of Indian-ASR-Bench, a reproducible WER benchmark for ASR on Indian English speech.
Usage
from transformers import pipeline
pipe = pipeline("automatic-speech-recognition", model="theshivam7/whisper-tiny-indian-english")
result = pipe("audio.wav")
print(result["text"])
For audio longer than 30 seconds, enable chunking:
result = pipe("long_audio.wav", chunk_length_s=30, stride_length_s=5)
Model description
A full fine-tune (all 39M parameters) of Whisper Tiny via transformers Seq2SeqTrainer,
using a step-based recipe (max_steps=2000, effective batch size 32, fp16, best-checkpoint
selection on validation WER). Best checkpoint: step 600/2000.
Intended uses & limitations
Evaluated on Indian-accented English academic lecture speech (TIE_shorts). Not evaluated on other English varieties or domains.
⚠️ Speaker overlap (disclosed). The training data's official split shares speakers with the test split (100% of test speakers, and 100% of test clips, are from speakers also seen in training). There is no clip-level leakage, but part of the measured gain over the pretrained baseline likely reflects speaker adaptation rather than purely accent/content adaptation. See the benchmark repo for full methodology.
⚠️ Gain is not statistically confirmed. The −2.96pp WER improvement over the pretrained baseline (below) does not survive Holm–Bonferroni correction across the 3-model fine-tuning family at 985 test clips (95% CI [−6.35, +0.13] pp, pHolm=0.195); read it as suggestive, not conclusive.
Training and evaluation data
- Train: TIE_shorts official train split, 7,200 filtered clips (46.9h).
- Eval: TIE_shorts official test split, 985 scored clips (one excluded for an empty reference).
Training procedure
| Base model | openai/whisper-tiny (39M) |
| Steps | 2000 (best checkpoint at step 600) |
| Effective batch size | 32 |
| Precision | fp16 |
| Optimizer | AdamW (transformers Seq2SeqTrainer default) |
| Selection | best checkpoint by validation WER |
Evaluation results
Corpus WER on TIE_shorts test split, transcript_clean normalization, both models decoded
through the identical HF transformers pipeline (engine-controlled comparison):
| Model | Corpus WER |
|---|---|
| openai/whisper-tiny (pretrained baseline) | 22.10% |
| theshivam7/whisper-tiny-indian-english (this model) | 19.14% |
Δ = −2.96 pp (95% CI [−6.35, +0.13], pHolm=0.195, not significant after correction, see limitations above).
Framework versions
- Transformers, PyTorch, Datasets (see training environment)
Citation
Part of Indian-ASR-Bench. See the repo for full benchmark methodology, error analysis, and reproduction instructions.