Whisper-Tiny-MLA (24 languages) — on-device-tier MLA-Whisper, 62.5% smaller decode KV-cache
openai/whisper-tiny (39M) with decoder self-attention converted MHA→MLA
(Whisper-MLA, arXiv:2603.00563), recovery-fine-tuned on a
24-language application set — the smallest, most deployable tier of the 24-language line.
from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-tiny-mla-24lang", trust_remote_code=True) # transformers==4.46.x
Honest sizing note (read first)
Conversion cost GROWS as the model shrinks — measured across the 24-language family: small ≈+0.6 → base ≈+1.1 → tiny ≈+1.9 (approximate per-language medians). At the tiny tier you pay ≈+1.9 WER (median) for the 62.5% cache cut, and absolute quality is tiny-tier (much higher WER than small — that is the base model, not MLA). If quality is the priority prefer whisper-small-mla-24lang; this tier is for memory-constrained deployment where the cache cut matters most.
Results (CommonVoice-17 test — Malay from FLEURS, n≤1500/lang (ko 339, no 370, ms 749, vi 1274; rest 1500); cost = paired vs an identically-trained control; CER for th/zh/ja)
| lang | WER% | CER% | conversion cost (WER pts) | sig |
|---|---|---|---|---|
| en | 29.8 | 16.0 | 1.69 | ns |
| de | 44.1 | 17.0 | 1.97 | sig |
| es | 30.9 | 11.2 | 1.39 | sig |
| fr | 47.1 | 20.9 | 2.38 | sig |
| it | 45.1 | 13.8 | 1.32 | sig |
| pt | 44.7 | 17.3 | 1.65 | ns |
| ru | 43.9 | 14.6 | 3.28 | sig |
| nl | 40.5 | 16.1 | 1.76 | sig |
| pl | 49.6 | 16.0 | 2.33 | sig |
| id | 54.9 | 20.1 | 1.94 | ns |
| tr | 54.2 | 17.9 | 2.17 | sig |
| hi | 47.6 | 25.1 | 1.66 | sig |
| ms | 51.1 | 19.2 | 1.64 | sig |
| sv-SE | 60.5 | 24.9 | 2.57 | sig |
| th | 83.8 | 32.5 | 0.43 | ns |
| zh-CN | 55.6 | 34.5 | -0.23 | ns |
| cs | 69.6 | 22.0 | 2.39 | sig |
| vi | 56.3 | 29.3 | 1.94 | sig |
| fi | 69.5 | 18.2 | 2.57 | sig |
| el | 69.0 | 28.2 | 5.37 | sig |
| da | 75.8 | 33.8 | 2.05 | ns |
| ja | n/a | 48.3 | -0.50 | ns |
| nn-NO | 82.9 | 34.6 | -10.16 | ns |
| ko | 78.0 | 44.3 | 6.61 | sig |
Thin-CommonVoice languages (Korean, Norwegian, Danish, Greek, Finnish, Czech, Vietnamese) are the weakest — a DATA limit, not an MLA one. At this tiny tier several are FLOOR cells (WER 69–83%: nn-NO 82.9, ko 78.0, da 75.8, cs 69.6, fi 69.5, el 69.0) — reported for transparency, NOT usable, exactly like Georgian at small sizes. At that error level the paired "conversion cost" is metric noise, not an MLA effect (e.g. nn-NO −10.16 just means one near-garbage arm hallucinated less on a 370-clip pool) — so the size-cost curve applies only to the languages where both arms are actually usable, not to these floor cells. Per-language cost also depends on the recovery mix: a language's cost here differs from the same language in whisper-tiny-mla-cv11 (the 11-language model) — expected (mix composition moves each language's low-rank fit), not irreproducibility. Malay was trained on FLEURS (no CV Malay exists; FLEURS is CC-BY-4.0; its eval-only license respected — audio used for training, never redistributed).
Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-tiny-24lang (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.
Limitations
- What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
- Runtime: requires
trust_remote_code+ transformers==4.46.x (no whisper.cpp/CT2/faster-whisper). - Language coverage: covers these 24 languages; erodes Whisper's others.
- Decoding: greedy-decode evals (beam-5 adds ~1-2 WER on both arms; conversion cost unchanged — measured).
- Domain: consumer-mic read-speech.
- Training: 15k steps, warmup+cosine, encoder frozen both arms, dev-selected, bf16 (weights released as fp16).