Whisper-Large-v3-MLA (11 languages) — compression cost ≈ statistical zero
openai/whisper-large-v3 with decoder self-attention converted MHA→MLA
(Whisper-MLA, arXiv:2603.00563) and recovery-fine-tuned on the
CC0 Whispered corpus — the same
model that annotated that corpus, now with a 62.5% smaller decode KV-cache. Conversion cost vs
an identically-trained unconverted control: not statistically significant on 9 of 11 languages
(significant only on ar +0.56 WER and zh +0.52 CER; median ≈ +0.1). n=1500/lang, paired per-speaker bootstrap.
from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-large-v3-mla-cv11", trust_remote_code=True) # transformers==4.46.x
Results (CommonVoice-17 test, greedy, Whisper normalization + Arabic folding)
| Lang | WER/CER % | conversion cost |
|---|---|---|
| en | 6.6 / 3.2 | +0.20 ns |
| de | 5.3 / 1.9 | +0.09 ns |
| es | 4.2 / 1.4 | +0.02 ns |
| fr | 9.6 / 3.6 | +0.08 ns |
| ru | 4.2 / 1.1 | −0.03 ns |
| tr | 8.4 / 1.9 | −0.08 ns |
| cy | 35.8 / 12.0 | +0.67 ns |
| ar | 16.6 / 6.0 | +0.56 ✱ |
| th | 14.0 / 4.4 | −0.02 CER ns |
| zh | 18.3 / 9.7 | +0.52 CER ✱ |
| ka | 72.3 / 34.3 | +0.22 ns — floor |
Size-scaling (measured across the released family): conversion cost falls with model size — tiny ≈+1.9 → base ≈+1.0 → small ≈+0.4 → medium ≈+0.4 → large ≈ 0 (approximate per-size medians; the exact per-language costs are in each model's table). We preregistered predictions twice and revised twice (decay → "saturation" → decay-to-zero); the monotone decline is the honest result.
Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-large-v3-cv11 (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.
Limitations
- Costs are recovery-mix-specific: do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (French in the small model: +0.32 cv11 vs +1.22 24-lang). Not irreproducibility.
- What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
- Requires
trust_remote_code=Trueandtransformers==4.46.x(custom MLA attention modules); not loadable in whisper.cpp / faster-whisper / CTranslate2. - Language coverage is the 11 recovery languages — unseen scripts degrade (measured); ka reported but weak at every size.
- Consumer-mic read-speech domain (SNR-ladder at small: cost flat to 0 dB); greedy-decode evals — beam-5 adds ~1-2 WER on both arms without changing the conversion cost (measured on the small tier).
- Training: 15k steps, warmup+cosine, encoder frozen both arms, dev-selected, bf16; released weights fp16 (~3.1 GB).
FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)
ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified to be small on the small tier). Per-language absolute: en 8.0 · de 7.2 · es 5.8 · fr 8.5 · ru 9.5 · tr 10.8 · cy 33.5 · ar 13.2 · th 17.9 · zh-CN 10.0 · ka 87.7. (Full JSON: the release repo's eval artifacts.)