european_accent_in_english_classification

A 16-class classifier of the speaker's first language (L1) from English speech, built by fully fine-tuning Orange/Speaker-wavLM-pro (WavLM-large, 316 M parameters) and trained to be invariant to the transmission channel, so that it stays usable on narrowband radio audio.

Replaces an earlier 15-class model (frozen backbone, MLP head) on this repo.

Labels (16)

14 European L1s — Croatian Czech Dutch Estonian Finnish French German Hungarian Italian Lithuanian Polish Romanian Slovak Spanish — plus Indian and Vietnamese, both common on European frequencies. Slovene from the previous version is gone: it is absent from the intersection of the two training corpora.

Evaluation — unseen speakers only

Every split is speaker-disjoint: no speaker of a test set appears in training, so these numbers measure generalisation to a voice never heard, not voice recognition.

Averaging logits over several clips of the same voice is worth ~14 points (0.742 → 0.881): do it whenever you have more than one utterance per speaker.

The previous model on this repo reported 66.98 % per clip / 85.71 % per speaker on 15 speaker-disjoint European classes. The label sets differ (16 vs 15, Indian and Vietnamese added, Slovene removed) and the splits were redrawn, so the two are of the same order but not strictly comparable.

Channel invariance

A classifier trained on clean wideband speech collapses on VHF radio. Measured on a paired corpus (identical voices, identical texts, with and without a simulated VHF channel), the predicted label distribution moved by JSD 0.541 — the prediction tracked the channel, not the speaker, collapsing onto a couple of attractor classes.

Training each crop as two views — clean, and passed through a randomly drawn VHF channel (PSD noise from 120 real ATC channel profiles, SNR −10…+20 dB, soft-clip drive 1–6, Butterworth 180–420 / 2500–3800 Hz) — under a consistency loss 1 − cos(e_clean, e_channel) on the embeddings brings that down to JSD 0.068.

Read this carefully: the invariance is distributional, not per-utterance. Corpus-level shares are stable across channels; an individual label still flips roughly seven times out of ten when the channel changes. Use this model to compare populations, not to label a single clip.

Validation on real air-traffic-control audio

Accent is not annotated in any public ATC corpus, so two proxies were used.

Controllers (ATCO2). A tower controller's L1 is almost certainly the local language, and the speaker_label field isolates the 449 controller utterances. The model agrees with the station country 43.0 % of the time — 7× the 6.2 % chance level — and 46.5 % when Czech and Slovak (mutually intelligible, and the model's main confusion) are merged.

Swiss contrast (LiveATC), the cleanest test. Zurich/Bern/Grenchen are German-speaking, Geneva/Payerne French-speaking, but all are captured on the same network with the same feed types — so the channel is held fixed and only the linguistic region changes. Over 5 276 clips:

Training

Two corpora pooled on their 16 common L1 classes, 1 078 distinct speakers (681 from the SAA, where one recording is one speaker, and 397 from the English corpus):

• Speech Accent Archive (GMU) — read speech, one speaker per recording, many voices;
• westbrook/English_Accent_DataSet — spontaneous speech, many clips per speaker. Its shipped splits are not speaker-disjoint (all 250 test speakers also appear in train), so they are discarded and the split is redrawn by speaker.

Mixing the two matters more than volume: a speaker exists in only one of them, so memorising voices stops paying and the model has to rely on what they share. Trained on the English corpus alone, the same backbone scored 0.351 on unseen SAA speakers; pooled, it reaches 0.835.

Settings: 4 s crops, speed perturbation 0.9–1.1, sampler weighted 1/n^0.5 over classes and corpora, AdamW (backbone 1e-5, head 1e-3), cosine schedule, bf16, gradient checkpointing, 30 epochs, checkpoint chosen on the mean of the two corpora's macro accuracy.

Architecture

WavLM-large (from Orange/Speaker-wavLM-pro)
  → learned softmax-weighted sum of the 25 hidden states   [added]
  → mean + std statistics pooling
  → TopLayers (2048 → 512 → 250), L2-normalised embedding
  → AAM-softmax head (m=0.2, s=30), 16 classes             [added]

Accent is a phonetic cue carried by intermediate WavLM layers, not only the last one, hence the learned layer weighting; it is initialised so the model starts out reproducing the original Orange embedding (verified: cosine = 1.000 against the reference stack).

Usage

from modeling_accent import AccentClassifier, predict

model = AccentClassifier.from_pretrained_hub(device="cuda")
print(predict("utterance.wav", model))
# {'accent': 'French', 'confidence': 0.94, 'probs': {...}}

Audio must be 16 kHz mono. predict averages logits over sliding 4 s windows covering the utterance. For several clips of one speaker, average the logits across clips before the argmax.

Limitations

• Per-utterance labels are unreliable across channels (28.6 % agreement, see above). Aggregate.
• The model is over-confident when wrong: mean confidence 0.94 when correct against 0.86 when wrong. Confidence is not a usable reliability signal — calibration was not addressed.
• Czech ↔ Slovak is the dominant confusion, and it is linguistic rather than accidental.
• Trained on 4 s crops; utterances under 2 s are out of distribution.
• The label set is not purely European despite the repo name (Indian, Vietnamese included).

Files