omniASR-CTC-300M-v2

Wav2Vec2 CTC ASR model (v2) converted from the OmniLingual fairseq2 checkpoint omniASR_CTC_300M_v2.

This model outputs CTC logits over a SentencePiece vocabulary and can transcribe speech in multiple languages.

Code Base

The code base for the conversion can be found here.

Usage

from transformers import Wav2Vec2ForCTC, AutoProcessor
import torch, torchaudio
processor = AutoProcessor.from_pretrained("aadel4/omniASR-CTC-300M-v2")
model     = Wav2Vec2ForCTC.from_pretrained("aadel4/omniASR-CTC-300M-v2")
model.eval()
waveform, sr = torchaudio.load("audio.wav")
if sr != 16_000:
    waveform = torchaudio.functional.resample(waveform, sr, 16_000)
inputs = processor(
    waveform.squeeze().numpy(), sampling_rate=16_000, return_tensors="pt"
)
with torch.no_grad():
    logits = model(**inputs).logits          # (1, T, vocab)
pred_ids   = torch.argmax(logits, dim=-1)
transcript = processor.decode(pred_ids[0])
print(transcript)