Whisper Large V3 Turbo - Persian Fine-Tuned
This repository contains a Persian (Farsi) fine-tuned version of OpenAI Whisper Large V3 Turbo for automatic speech recognition (ASR).
Model Information
- Model:
sadrasa97/whisper-large-v3-turbo-fa - Base Model:
openai/whisper-large-v3-turbo - Task: Automatic Speech Recognition
- Language: Persian / Farsi
- Language Code:
fa - Architecture: Whisper
- Framework: Hugging Face Transformers
Installation
pip install -U torch transformers accelerate librosa soundfile
## Quick Start
The recommended way to use the model is with the Hugging Face `pipeline` API.
```python
import torch
from transformers import (
AutoModelForSpeechSeq2Seq,
AutoProcessor,
pipeline,
)
model_id = "sadrasa97/whisper-large-v3-turbo-fa"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = (
torch.float16
if torch.cuda.is_available()
else torch.float32
)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id,
torch_dtype=torch_dtype,
low_cpu_mem_usage=True,
)
model.to(device)
processor = AutoProcessor.from_pretrained(
model_id
)
pipe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
torch_dtype=torch_dtype,
device=0 if torch.cuda.is_available() else -1,
)
result = pipe(
"audio.wav",
generate_kwargs={
"language": "fa",
"task": "transcribe",
},
)
print(result["text"])
GPU Inference
For NVIDIA CUDA GPUs, FP16 inference can be used to reduce memory consumption and improve inference performance.
import torch
from transformers import (
AutoModelForSpeechSeq2Seq,
AutoProcessor,
pipeline,
)
model_id = "sadrasa97/whisper-large-v3-turbo-fa"
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id,
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
)
model.to("cuda")
processor = AutoProcessor.from_pretrained(
model_id
)
pipe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
torch_dtype=torch.float16,
device=0,
)
result = pipe(
"audio.wav",
generate_kwargs={
"language": "fa",
"task": "transcribe",
},
)
print(result["text"])
CPU Inference
The model can also be used on CPU.
import torch
from transformers import (
AutoModelForSpeechSeq2Seq,
AutoProcessor,
pipeline,
)
model_id = "sadrasa97/whisper-large-v3-turbo-fa"
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id,
torch_dtype=torch.float32,
low_cpu_mem_usage=True,
)
processor = AutoProcessor.from_pretrained(
model_id
)
pipe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
torch_dtype=torch.float32,
device=-1,
)
result = pipe(
"audio.wav",
generate_kwargs={
"language": "fa",
"task": "transcribe",
},
)
print(result["text"])
Direct Model Usage
For applications requiring lower-level control over preprocessing and generation, the model can be loaded directly.
import torch
import librosa
from transformers import (
AutoModelForSpeechSeq2Seq,
AutoProcessor,
)
model_id = "sadrasa97/whisper-large-v3-turbo-fa"
device = "cuda" if torch.cuda.is_available() else "cpu"
torch_dtype = (
torch.float16
if device == "cuda"
else torch.float32
)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id,
torch_dtype=torch_dtype,
low_cpu_mem_usage=True,
)
model.to(device)
model.eval()
processor = AutoProcessor.from_pretrained(
model_id
)
audio, sample_rate = librosa.load(
"audio.wav",
sr=16000,
)
inputs = processor(
audio,
sampling_rate=sample_rate,
return_tensors="pt",
)
input_features = inputs.input_features.to(device)
with torch.inference_mode():
predicted_ids = model.generate(
input_features,
language="fa",
task="transcribe",
)
transcription = processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)
print(transcription[0])
Persian Transcription
For Persian speech recognition, explicitly specify the language as Persian:
generate_kwargs = {
"language": "fa",
"task": "transcribe",
}
The task should be set to transcribe when the goal is to produce text in the original spoken language.
For example:
result = pipe(
"persian_audio.wav",
generate_kwargs={
"language": "fa",
"task": "transcribe",
},
)
print(result["text"])
Audio Input
Whisper operates on audio sampled at 16 kHz internally.
For best results, use:
- 16 kHz sampling rate
- Mono audio
- Clear speech
- Minimal background noise
- Minimal reverberation
The Hugging Face processor handles the required feature extraction.
Supported Use Cases
This model can be used for:
- Persian speech recognition
- Farsi speech-to-text
- Persian audio transcription
- Voice assistants
- Conversational AI
- Voice-to-voice systems
- Customer support applications
- Persian NLP pipelines
- Speech analytics
- Meeting transcription
- Audio search
- Research and experimentation
Model Architecture
The model is based on the Whisper Large V3 Turbo architecture.
Whisper is an encoder-decoder Transformer architecture designed for multilingual speech recognition and related speech-processing tasks.
This model has been fine-tuned for improved performance on Persian/Farsi speech.
Fine-Tuning
Base Model
openai/whisper-large-v3-turbo
Target Language
Persian / Farsi
Language Code
fa
The fine-tuning process adapts the pretrained Whisper model to Persian speech characteristics, vocabulary, pronunciation patterns, and linguistic structure.
Evaluation
Recommended evaluation metrics for Persian ASR include:
- Word Error Rate (WER)
- Character Error Rate (CER)
- Normalized WER
- Normalized CER
For Persian ASR, normalization is particularly important because Unicode variants, Arabic/Persian character differences, whitespace, punctuation, and zero-width non-joiner usage can significantly affect raw WER and CER.
When reporting performance, evaluation should preferably be performed using a held-out test set representative of the intended deployment domain.
Limitations
ASR performance depends on the characteristics of the input audio and the target domain.
Potential sources of transcription errors include:
- Background noise
- Low-quality recordings
- Strong reverberation
- Regional accents and dialects
- Very fast speech
- Multiple speakers
- Overlapping speech
- Domain-specific terminology
- Uncommon names and words
- Code-switching between Persian and other languages
The model should be evaluated on representative production data before deployment in safety-critical or high-impact applications.
Production Considerations
For production deployment, consider:
- GPU inference for lower latency
- FP16 inference on compatible NVIDIA GPUs
- Audio preprocessing and normalization
- Voice Activity Detection (VAD)
- Chunking long audio files
- Batch inference for higher throughput
- Appropriate timeout and memory limits
- Domain-specific evaluation
- WER/CER monitoring
- Post-processing and Persian text normalization
For real-time applications, streaming audio should be segmented into appropriate chunks and processed incrementally rather than waiting for the complete recording.
Transformers Example
The model can be loaded directly through the standard Transformers APIs:
from transformers import (
AutoModelForSpeechSeq2Seq,
AutoProcessor,
)
model_id = "sadrasa97/whisper-large-v3-turbo-fa"
processor = AutoProcessor.from_pretrained(
model_id
)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id
)
Base Model
This model is based on:
openai/whisper-large-v3-turbo
Original base model:
https://huggingface.co/openai/whisper-large-v3-turbo
Citation
If you use this model in academic research, please cite the original Whisper publication:
@misc{radford2022robust,
title={Robust Speech Recognition via Large-Scale Weak Supervision},
author={
Alec Radford and
Jong Wook Kim and
Tao Xu and
Greg Brockman and
Christine McLeavey and
Ilya Sutskever
},
year={2022},
eprint={2212.04356},
archivePrefix={arXiv},
primaryClass={eess.AS}
}
License
This model is based on openai/whisper-large-v3-turbo.
Please review the license and usage terms of the original base model, as well as the licenses and terms associated with any datasets used during fine-tuning.
Author
Sadra Saremi
Hugging Face:
https://huggingface.co/sadrasa97