Whisper-Base-MLA (11 languages) — MLA-converted, 62.5% smaller decode KV-cache
The on-device-tier sibling of the WhisperMLA family: openai/whisper-base (74M) with its
decoder self-attention converted MHA→MLA (per Whisper-MLA, arXiv:2603.00563),
recovery-fine-tuned on 11 languages of the CC0
Whispered corpus (32k clips/lang).
from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-base-mla-cv11", trust_remote_code=True) # transformers==4.46.x
Honest sizing note (read this first)
Conversion cost grows as the student shrinks — measured across the family: small ≈ +0.4
median WER → base ≈ +1.0 → tiny ≈ +1.9. At the base tier you pay ≈ +1.0 WER (median) for the
62.5% cache cut. If quality is the priority, prefer the small variant; this tier is for
memory-constrained deployments where the cache cut matters most.
Results (CommonVoice-17 test, n=1500/lang; WER/CER %; cost = paired vs an identically-trained unconverted control)
| Lang | this model (WER / CER) | conversion cost |
|---|---|---|
| en | 21.1 / 11.2 | +1.04 ✱ |
| de | 30.1 / 11.3 | +0.83 (ns) |
| es | 19.0 / 6.7 | +0.88 ✱ |
| fr | 32.3 / 13.9 | +0.59 (ns) |
| ru | 28.8 / 8.7 | +1.31 ✱ |
| tr | 38.9 / 11.8 | +1.13 ✱ |
| cy | 79.9 / 34.2 | +2.10 ✱ |
| ar | 58.9 / 24.5 | +2.71 ✱ |
| th | 62.3 / 22.3 | +0.24 CER (ns) |
| zh | 41.8 / 24.6 | −0.27 CER (ns) |
| ka | 111.8 / 75.8 | +12.74 ✱ — floor |
Absolute quality is base-tier-typical (much lower than small — that is the base model, not
MLA). Encoder frozen both arms; 15,000 steps; warmup+cosine; bf16 (weights released as fp16).
Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-base-cv11 (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.
Limitations
- Costs are recovery-mix-specific: do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (French in the small model: +0.32 cv11 vs +1.22 24-lang). Not irreproducibility.
- What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
- Requires
trust_remote_code=Trueandtransformers==4.46.x(custom MLA attention modules); not loadable in whisper.cpp / faster-whisper / CTranslate2. - Language coverage is the 11 recovery languages — unseen scripts degrade.
- Georgian (ka) reported as the labeled model-class floor; not usable at this tier.
- Read-speech domain, greedy-decode evals — beam-5 adds ~1-2 WER on both arms without changing the conversion cost (measured on the small tier).
Acoustic conditions of the evaluation
Evaluated on crowdsourced consumer-microphone recordings with real environmental noise — traffic, room reverb, variable devices — CommonVoice's native conditions, not studio audio. The numbers above already include that heterogeneity. Not yet benchmarked: far-field, telephony (8 kHz), overlapping speech; an SNR-ladder robustness section will be added when measured.
FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)
ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified to be small on the small tier). Per-language absolute: en 16.1 · de 25.8 · es 15.3 · fr 30.4 · ru 30.5 · tr 36.0 · cy 82.0 · ar 55.4 · th 34.7 · zh-CN 23.5 · ka 100.2. (Full JSON: the release repo's eval artifacts.)