waxal-whisper-small-spec_aug_v1
Fine-tuned openai/whisper-small on
Google WaxalNLP
(Lingala, Shona, Luganda) with SpecAugment on log-mel features.
Training champion from the whisper-small exploration
(Phase-1 holdout score 0.2670 = 0.5·WER + 0.5·CER).
Continued / decode-focused variants (e.g. freeze-encoder + higher LR) were
explored on top of this recipe; see the sibling base card
mmwanje/waxal-whisper-small.
Training notes
- Joint multilingual FT (no language tokens in labels — better for Phase 2)
- SpecAugment applied on padded mel batches at train time
- 2×P100 DDP; typical recipe shape:
batch=2,grad_accum=4, ~800 steps (see repo experiment log) - Base init:
openai/whisper-small(or continue frommmwanje/waxal-whisper-small)
Usage
from transformers import WhisperProcessor, WhisperForConditionalGeneration
repo = "mmwanje/waxal-whisper-small-spec_aug_v1"
processor = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(repo)
| Code | Language | Whisper lang token |
|---|---|---|
| lin | Lingala | lingala |
| sna | Shona | shona |
| lug | Luganda | swahili (closest) |
Phase 2
Unlabeled clips: run MMS LID (facebook/mms-lid-4017) → force Whisper into
{lingala, shona, swahili}. Decode sweeps (no_cond, lang prompts, beams)
are inference-only — no retrain required.