Welsh ASR — XLS-R 300m fine-tuned on FLEURS Welsh
facebook/wav2vec2-xls-r-300m fine-tuned for Welsh speech recognition with a character-level CTC head.
Results
FLEURS Welsh test set, 1021 utterances. Zero-shot Whisper-small is scored on the same audio through an identical text-normalization path.
| Model | WER ↓ | CER ↓ | WER, no digits | CER, no digits |
|---|---|---|---|---|
| Whisper-small (zero-shot) | 0.5987 | 0.2289 | 0.5853 | 0.2149 |
| This model | 0.3991 | 0.1141 | 0.3879 | 0.1098 |
| relative change | −33.3% | −50.1% | −33.7% | −48.9% |
About a fifth of FLEURS utterances contain numerals, which a character-level CTC model cannot produce from audio, so results are given both overall and on the digit-free subset. Both models are always scored on the same subsets.
Usage
from transformers import pipeline
pipe = pipeline("automatic-speech-recognition",
model="pnawani/welsh-asr-xlsr-300m")
print(pipe("audio.wav")["text"])
The model expects 16 kHz mono audio. Output is lowercase, without punctuation,
and keeps the Welsh circumflex vowels âêîôŵŷ.
Training
| Data | FLEURS Welsh, 11.3 h after filtering (2784 clips) |
| Steps | 2100 (~12 epochs) |
| Batch | 2 × 8 gradient accumulation (effective 16) |
| Learning rate | 3e-4, 200 warmup steps, fp16 |
| Hardware | one free Colab T4 |
Two filters matter. Clips over 30 s are dropped, costing 1.7% of the data; the 20 s cutoff common in XLS-R tutorials would discard 31%, as FLEURS utterances are unusually long. Separately, ~15% of the train split pairs a truncated clip with its full transcript — in the worst case 0.96 s of audio against 303 characters. wav2vec2 downsamples by 320, so CTC cannot emit those labels and returns infinite loss, which becomes NaN gradients within two steps. Those clips are short, so removing them costs under 1 of 12.2 hours.
Text is lowercased, apostrophes deleted (mae'r → maer) and punctuation
stripped. Welsh circumflex vowels are kept; other diacritics are folded to their
base letters. The vocabulary is 45 characters.
Limitations
Numerals. The model cannot transcribe spoken numbers as digits — there is no
acoustic evidence mapping dwy fil ar bymtheg to 2019. Roughly 19% of the test
set contains numerals.
Circumflex vowels. Only 57% of the circumflex vowels present in references are
produced (259 of 457); â → a accounts for 147 substitutions. The circumflex
marks vowel length in Welsh (tan "fire" vs tân "until"), which is acoustically
subtle and often recoverable only from context. A CTC decoder with no language
model has no mechanism for this.
Vowel confusion is the dominant error. Fine-tuning halved character
substitutions (14,032 → 6,681), but vowel-for-vowel confusion still accounts for
41% of them — essentially unchanged from the baseline's 41.3%. Welsh y, u and
i are acoustically close and map onto no single English vowel.
No language model. Decoding is plain CTC argmax. A KenLM decoder would likely recover a substantial part of both the circumflex and vowel-confusion errors.
Domain. FLEURS is read speech from Wikipedia-style text. Expect degradation on spontaneous or conversational Welsh, and on dialects under-represented in FLEURS.
Code
Training, evaluation and error-analysis code: https://github.com/paarthN/welsh-asr
Acknowledgments
FLEURS (Google), XLS-R (Meta AI), Hugging Face Transformers.