Whisper Medium Fine-tuned for Spontaneous Persian Speech
This model is a fine-tuned version of openai/whisper-medium
for Persian automatic speech recognition (ASR).
It was adapted on the Spontaneous Persian Speech (SPS) dataset, with a focus on spontaneous and conversational Persian.
Model description
- Base model:
openai/whisper-medium - Language: Persian (Farsi)
- Task: Automatic Speech Recognition
- Adaptation method: Full fine-tuning
- Training duration: 2 epochs
Training data
The model was trained on speech segments extracted from the Spontaneous Persian Speech (SPS) dataset.
For this training run:
- Training segments: 16,409
- Validation segments: 1,985
The SPS dataset itself is not distributed with this model.
Training configuration
- Learning rate: 5e-6
- Warm-up ratio: 0.05
- Batch size: 1
- Gradient accumulation steps: 64
- Effective batch size: 64
- Maximum target length: 160 tokens
- Gradient checkpointing: enabled
Evaluation
The saved evaluation run contains 500 scored test segments.
Mean segment-level results:
| Model | WER | CER |
|---|---|---|
| Whisper-medium baseline | 2.368 | 1.857 |
| Fine-tuned Whisper-medium | 0.768 | 0.506 |
These values are mean segment-level error rates rather than corpus-level aggregated WER/CER.
The fine-tuned model substantially improves over the corresponding Whisper-medium baseline on this evaluation set.
A separate phonological analysis was also conducted to examine how fine-tuning affected phone-substitution errors and their phonological characteristics.
Usage
The model can be loaded directly with Hugging Face Transformers and used to transcribe Persian audio.
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
model_id = "Behnooshn/whisper-medium-sps-persian"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id)
Intended use
This model is intended primarily for research on Persian ASR, particularly spontaneous and conversational Persian speech.
Performance on other speech domains, recording conditions, or Persian varieties has not been systematically evaluated.
Limitations
The model was fine-tuned on a limited spontaneous Persian dataset. Its performance should therefore not be assumed to generalize equally to all speakers, domains, dialects, or acoustic conditions.
Data availability
The SPS training data are not included in this repository.
Base model
This work builds on OpenAI Whisper:
openai/whisper-medium