Whisper-Medium-MLA (11 languages) — 62.5% smaller decode KV-cache
openai/whisper-medium (769M) with decoder self-attention converted MHA→MLA
(Whisper-MLA, arXiv:2603.00563), recovery-fine-tuned on 11
languages of the CC0 Whispered
corpus (32k clips/lang). The largest absolute cache savings of the released family — medium's
per-stream decode cache is ~2.7× small's, so the 62.5% cut buys the most bytes here. For the
family's top accuracy tier, see whisper-large-v3-mla.
from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-medium-mla-cv11", trust_remote_code=True) # transformers==4.46.x
Results (CommonVoice-17 test, n=1500/lang; cost = paired vs identically-trained unconverted control)
| Lang | this model (WER/CER %) | conversion cost | sig |
|---|---|---|---|
| en | 8.7 / 4.4 | +0.41 | ✱ |
| de | 9.1 / 3.2 | +0.39 | ✱ |
| es | 6.6 / 2.2 | +0.19 | ns |
| fr | 13.2 / 5.0 | −0.20 | ns |
| ru | 8.4 / 2.4 | +0.34 | ✱ |
| tr | 13.3 / 3.5 | +0.16 | ns |
| cy | 42.2 / 14.5 | +0.73 | ✱ |
| ar | 24.3 / 9.0 | +0.42 | ns |
| th | 19.8 / 6.2 | CER +0.08 | ns |
| zh | 19.8 / 10.7 | CER +0.53 | ✱ |
| ka † | 80.6 / 36.8 | +0.54 | ns |
Median cost ≈ +0.4; not significant on 6/11 languages (zh cost is significant on CER). † ka: the compression cost collapses to ns at medium, but ka itself stays unusable (80.6 WER; capacity was and is its problem).
The honest scaling note (preregistered prediction, half-held)
We predicted medium's cost at ≈+0.2 before training. Measured: ≈+0.4 — the size-scaling curve SATURATES (tiny ≈+1.9 → base ≈+1.0 → small ≈+0.4 → medium ≈+0.4; approximate per-size medians) rather than decaying to zero. The practical claim holds (all languages ≤+1.0, mostly ns); the mechanism reading is an irreducible truncation cost at this rank, not ever-growing redundancy absorption.
Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-medium-cv11 (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.
Limitations
- Costs are recovery-mix-specific: do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (French in the small model: +0.32 cv11 vs +1.22 24-lang). Not irreproducibility.
- What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
- Requires
trust_remote_code=Trueandtransformers==4.46.x(custom MLA attention modules); not loadable in whisper.cpp / faster-whisper / CTranslate2. - Language coverage is the 11 recovery languages — unseen scripts degrade (measured).
- Consumer-mic read-speech domain (real environmental noise included; SNR-ladder on the small tier showed cost flat to 0 dB); greedy-decode evals — beam-5 adds ~1-2 WER on both arms without changing the conversion cost (measured on the small tier).
- Training: 15k steps, warmup+cosine, encoder frozen both arms, dev-checkpoint-selected, bf16 (weights released as fp16).
FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)
ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified to be small on the small tier). Per-language absolute: en 9.6 · de 9.7 · es 7.3 · fr 11.0 · ru 12.6 · tr 14.4 · cy 41.4 · ar 20.8 · th 20.0 · zh-CN 11.1 · ka 92.4. (Full JSON: the release repo's eval artifacts.)