BuzzASR — Macedonian
A monolingual automatic speech recognition model for Macedonian, fine-tuned from openai/whisper-large-v3. Part of BuzzASR, a suite of 102 language-specialized ASR models (paper: arXiv:2609.09554, Findings of EMNLP 2026).
This model uses simple fine-tuning (Whisper's tokenizer, ASR fine-tuning only).
🏆 State-of-the-art (open-source). On the combined FLEURS + Common Voice test set, this model achieves the lowest CER of every open system we compare against: Whisper-large-v3, Omnilingual 1B/7B, MMS, Qwen3-ASR, and Cohere Transcribe.
Results (normalized CER / WER, %)
| Test set | CER | WER | Whisper-large-v3 (zero-shot) CER |
|---|---|---|---|
| FLEURS | 2.93 | 10.74 | 4.62 |
| Common Voice 25 | 0.44 | 1.85 | 6.48 |
| Combined | 1.31 | 4.94 | 5.82 |
~4.4x CER reduction over Whisper zero-shot on the combined test set.
Usage
import torch, torchaudio
from transformers import WhisperForConditionalGeneration, WhisperProcessor
model = WhisperForConditionalGeneration.from_pretrained("BuzzASR/macedonian", torch_dtype=torch.float16).to("cuda").eval()
proc = WhisperProcessor.from_pretrained("BuzzASR/macedonian")
wav, sr = torchaudio.load("audio.wav") # 16 kHz mono
feats = proc(wav[0], sampling_rate=16000, return_tensors="pt").input_features.to("cuda").half()
ids = model.generate(feats, num_beams=1, no_repeat_ngram_size=3, repetition_penalty=1.2)
print(proc.batch_decode(ids, skip_special_tokens=True)[0])
The language/task prompt is baked into the generation config, so no language= argument is needed.
Training data
FLEURS + Common Voice Corpus 25.0 (Mozilla, March 2025; https://commonvoice.mozilla.org/en/datasets), capped per the paper. Text-only data from the Goldfish corpus (Chang et al., 2026).
Limitations
Monolingual (Macedonian only). Evaluated on FLEURS / Common Voice test splits; other domains or dialects may differ.
Links & citation
- Paper: https://arxiv.org/abs/2609.09554 (Findings of EMNLP 2026)
- Project page: https://lemn-lab.github.io/buzz-asr/
- All models: https://huggingface.co/BuzzASR
@misc{buzzasr2026,
title = {BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models},
author = {Shivam Singh and Aditya Yadavalli and Catherine Arnett and Alex Warstadt},
year = {2026},
eprint = {2609.09554},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
note = {Findings of the Association for Computational Linguistics: EMNLP 2026},
url = {https://arxiv.org/abs/2609.09554}
}