omniASR_W2V_7B_SSL-ctc-char-pld_ceb
Cebuano speech recognition: ylacombe/omniASR_W2V_7B_SSL
finetuned on the Philippine Language Dataset
(PLD), read speech collected by the UP Diliman Digital Signal Processing
Laboratory. Part of the halohalo project.
In plain words
This model turns recorded Cebuano speech into text. Give it an audio clip (a WAV file, 16 kHz, one channel), and it returns what was said.
How good is it? Tested on sentences it had never seen, spoken by people it had never heard, it gets about 83 of every 100 characters and 51 of every 100 words right. Word accuracy is the one you will notice.
It was trained on clear, read speech recorded for a corpus. Expect worse results on conversations, phone calls, music in the background, or a speaker switching languages mid-sentence.
Not sure which model to pick? For Cebuano, use whisper-large-v3-pld-ceb-norm — it is the most accurate one this organisation has published. This one exists for comparison and research.
Method (for practitioners)
CTC head on ylacombe/omniASR_W2V_7B_SSL (char units, 65 of them) for 5000 steps on 24059 clips, frozen-disjoint split (speakers and prompts unseen in training). Part of the R1/R2 bake-off in docs/pld_sota_track.md. Trained on PLD, which is CC-BY-NC and research-only: this checkpoint is a research artifact regardless of the base model's licence. Scale did not help. On Cebuano this 7B encoder scores 17.02% CER, the 1B sibling 17.04% and whisper-large-v3 16.38%, same split, same 5000 steps. Trained with fp32 weights and bitsandbytes 8-bit Adam to fit one 96 GB card. Kept in case a better decoder than a linear CTC head can use what the encoder knows.
Evaluation
Frozen speaker- and prompt-disjoint split of PLD (splits/pld_*.json in the repo): no test speaker and no test sentence appears anywhere in training. Numbers on this split are not comparable with the in-domain figures on the dataset card, which share both, and are typically several times higher for that reason.
cer and wer are character and word error rates on the whole test split, hypothesis and reference both lowercased and whitespace-normalised. Accents and punctuation in the reference count as errors when missing, which is strict: PLD marks stress on about a third of words. The -norm variant of this model uses the other convention. eval_loss is the training objective on the test split.
| metric | value |
|---|---|
| loss | 0.8755 |
| cer | 0.1702 |
| wer | 0.4946 |
How to use
from transformers import AutoFeatureExtractor, Wav2Vec2ForCTC
import torch, json, soundfile as sf
from huggingface_hub import hf_hub_download
repo = "sapinsapin/omniASR_W2V_7B_SSL-ctc-char-pld_ceb"
extractor = AutoFeatureExtractor.from_pretrained(repo)
model = Wav2Vec2ForCTC.from_pretrained(repo).eval()
vocab = json.load(open(hf_hub_download(repo, "vocab.json")))
id2unit = {i: u for u, i in vocab.items()}
wav, sr = sf.read("clip.wav") # 16 kHz mono
x = extractor(wav, sampling_rate=16000, return_tensors="pt").input_values
with torch.no_grad():
ids = model(x).logits[0].argmax(-1).tolist()
out, prev = [], None # greedy CTC: collapse repeats, drop blank (0)
for i in ids:
if i != prev and i != 0:
out.append(id2unit[i])
prev = i
print("".join(out).replace("|", " ").strip())
A 4-gram word LM over PLD's training transcripts, fused with pyctcdecode,
lowers WER by a further 4-7 points (scripts/ctc_lm_eval.py in the repo).
Caveats
- Trained on read, prompted speech; accuracy drops on spontaneous or noisy audio.
- One corpus, one recording setup. Cross-corpus tests on Filipino showed large drops for models of this kind.
Licence
cc-by-nc-4.0. PLD is CC-BY-NC and research-only, so every model trained on it
inherits that regardless of the base model's own licence. The base model's
terms apply in addition.
Trained with finetune_asr.py from halohalo; the dataset adapter
normalises each corpus to (audio@16k, text, speaker_id).