Whisper
Whisper-base for automatic speech recognition (ASR) for the low-resourced Kazakh language. The model was fine-tuned on the Kazakh Speech Corpus 2 with over 1k hours of labelled data. The model achieved 15.36% WER on the test set.
Usage
This checkpoint is a Kazakh-only model, meaning it can be used only for Kazakh speech recognition.
Transcription
>>> from transformers import WhisperProcessor, WhisperForConditionalGeneration
>>> import librosa
>>> # load model and processor
>>> processor = WhisperProcessor.from_pretrained("akuzdeuov/whisper-base.kk")
>>> model = WhisperForConditionalGeneration.from_pretrained("akuzdeuov/whisper-base.kk")
>>> # load your audio
>>> audio, sampling_rate = librosa.load("path_to_audio", sr=16000)
>>> input_features = processor(audio, sampling_rate=sampling_rate, return_tensors="pt").input_features
>>> # generate token ids
>>> predicted_ids = model.generate(input_features)
>>> # decode token ids to text
>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=False)
>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
The context tokens can be removed from the start of the transcription by setting skip_special_tokens=True.
Long-Form Transcription
The Whisper model is intrinsically designed to work on audio samples of up to 30s in duration. However, by using a chunking
algorithm, it can be used to transcribe audio samples of up to arbitrary length. This is possible through Transformers
pipeline
method. Chunking is enabled by setting chunk_length_s=30 when instantiating the pipeline. With chunking enabled, the pipeline
can be run with batched inference.
>>> import torch
>>> from transformers import pipeline
>>> device = "cuda:0" if torch.cuda.is_available() else "cpu"
>>> pipe = pipeline(
>>> "automatic-speech-recognition",
>>> model="akuzdeuov/whisper-base.kk",
>>> chunk_length_s=30,
>>> device=device,
>>> )
>>> prediction = pipe("path_to_audio", batch_size=8)["text"]