omniASR-CTC-300M-v2-Zulu
Wav2Vec2 CTC ASR model (Zulu) converted from the OmniLingual fairseq2 checkpoint omniASR_CTC_300M_v2.
Fine-tuned from facebook/omniASR-CTC-300M for isiZulu speech recognition on all isiZulu ASR training data in our mix, then converted to Hugging Face transformers format.
Best validation WER: 27.25 (step 38,500).
The model outputs CTC logits over a SentencePiece vocabulary.
Model details
| Property | Value |
|---|---|
| HF class | Wav2Vec2ForCTC |
| Encoder layers | 24 |
| Hidden size | 1024 |
| Attention heads | 16 |
| FFN intermediate | 4096 |
| Vocabulary size | 10288 |
| Source framework | fairseq2 |
| Source card | omniASR_CTC_300M_v2 |
| Parity verification | ✅ Verified |
Numerical parity against the original fairseq2 checkpoint has been confirmed: outputs match to within atol=1e-4 on a held-out audio sample.
Usage
from datasets import load_dataset, Audio
from transformers import Wav2Vec2ForCTC, AutoProcessor
import pandas as pd
import torch
MODEL_PATH = "uctnlp/omniASR-CTC-300M-v2-Zulu-Baseline"
DATASET_NAME = "danielshaps/nchlt_speech_zul"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# Load model
processor = AutoProcessor.from_pretrained(MODEL_PATH)
model = Wav2Vec2ForCTC.from_pretrained(MODEL_PATH)
model.to(device)
model.eval()
# Load NCHLT isiZulu test set
dataset = load_dataset(DATASET_NAME, split="test")
dataset = dataset.cast_column("audio", Audio(sampling_rate=16_000))
sample = dataset[0]
audio = sample["audio"]
inputs = processor(
audio["array"],
sampling_rate=16_000,
return_tensors="pt",
).to(device)
with torch.inference_mode():
logits = model(**inputs).logits
pred_ids = torch.argmax(logits, dim=-1)
transcript = processor.batch_decode(pred_ids)[0]
print("Reference :", sample["text"])
print("Prediction:", transcript)
# Expected results for test sample
# Reference : izokwenza umyalelo wokuthi
# Prediction: izokwenza umyalelo wokuthi