Whisper-Small-MLA (24 languages) — MLA-converted Whisper, 62.5% smaller decode KV-cache
openai/whisper-small (244M) with decoder self-attention converted MHA→MLA
(Whisper-MLA, arXiv:2603.00563), recovery-fine-tuned on a
24-language application set. Same 62.5% decode-cache cut as the rest of the family, on a wider
language span. Conversion cost vs an identically-trained unconverted control: not statistically
significant on 15 of 24 languages (n≤1500/lang (ko 339, no 370, ms 749, vi 1274; rest 1500), paired per-speaker bootstrap).
from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-small-mla-24lang", trust_remote_code=True) # transformers==4.46.x
Results (CommonVoice-17 test — Malay from FLEURS, n≤1500/lang (ko 339, no 370, ms 749, vi 1274; rest 1500); CER is the metric for th/zh/ja)
| lang | WER% | CER% | conversion cost (WER pts) | sig |
|---|---|---|---|---|
| en | 12.9 | 6.7 | 0.72 | sig |
| de | 15.5 | 5.5 | 0.59 | sig |
| es | 10.0 | 3.3 | -0.01 | ns |
| fr | 20.3 | 8.0 | 1.22 | sig |
| it | 15.1 | 4.2 | -0.25 | ns |
| pt | 15.1 | 5.3 | 0.32 | ns |
| ru | 14.7 | 4.2 | 0.69 | sig |
| nl | 13.6 | 4.4 | 0.42 | ns |
| pl | 17.8 | 5.0 | 0.50 | ns |
| id | 20.2 | 6.6 | 0.69 | ns |
| tr | 22.0 | 6.0 | 0.89 | sig |
| hi | 24.2 | 12.0 | 0.96 | sig |
| ms | 21.0 | 7.6 | 0.94 | sig |
| sv-SE | 22.9 | 8.0 | 0.93 | sig |
| th | 37.4 | 12.4 | 0.25 | ns |
| zh-CN | 28.5 | 15.5 | -0.08 | ns |
| cs | 27.9 | 7.2 | 0.54 | ns |
| vi | 28.6 | 13.6 | 0.28 | ns |
| fi | 28.8 | 5.6 | 0.34 | ns |
| el | 32.8 | 11.3 | 2.34 | sig |
| da | 32.9 | 12.0 | 0.59 | ns |
| ja | n/a | 24.3 | 1.01 | ns |
| nn-NO | 44.4 | 16.0 | 1.29 | ns |
| ko | 45.6 | 18.3 | 1.34 | ns |
Read this before deploying — data-driven quality tiers
The 24 languages are NOT equal in training data, and it shows in the absolute WER (the compression cost is small/ns almost everywhere — quality is a data story, not an MLA story):
- Well-resourced (usable): es 10, en 12.9, nl 13.6, ru 14.7, it/pt 15.1, de 15.5 WER; th CER 12.4, zh CER 15.5.
- Thin CommonVoice data (weaker, use with care): Korean 45.6, Norwegian 44.4, Danish 32.9, Greek 32.8, Finnish 28.8, Vietnamese 28.6, Czech 27.9 WER — single-shard languages in CV17.
- Malay (ms, 21.0 WER) was trained on FLEURS (no CommonVoice Malay exists; FLEURS is CC-BY-4.0); its eval-only license respected — the FLEURS audio is used for training here, never redistributed.
- Per-language conversion cost depends on the recovery mix: a shared language's cost here differs from the same language in whisper-small-mla-cv11 (the 11-language model) — e.g. French +1.22 here vs +0.32 there. Expected (mix composition moves each language's low-rank fit), not irreproducibility. Sub-zero costs on the CER-scored spaceless languages (zh) are within metric noise, not an MLA gain.
Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-small-24lang (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.
Limitations
- What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
- Runtime: requires
trust_remote_code+ transformers==4.46.x (no whisper.cpp/CT2/faster-whisper). - Language coverage: covers these 24 languages; erodes Whisper's others (unseen scripts most).
- Decoding: greedy-decode evals (beam-5 adds ~1-2 WER on both arms; conversion cost unchanged — measured).
- Domain: consumer-mic read-speech.
- Compression cost: ≈ small-tier — ≈+0.6 median across all 24 languages here (≈+0.4 across the well-resourced subset). For reference, the 11-language cv11 family cost curve is tiny ≈+1.9 → base ≈+1.0 → small ≈+0.4 → medium ≈+0.4 → large ≈0 (approximate per-size medians).
- Training: 15k steps, warmup+cosine, encoder frozen both arms, dev-selected, bf16 (weights released as fp16).