Whisper Medium — Kreol Morisien (In-Domain Fine-Tuned, With AC, 10h)
This model is a fine-tuned version of openai/whisper-medium on 10 hours of in-domain Kreol Morisien (Mauritian Creole) audio data, with accent conditioning.
Model Details
| Parameter | Value |
|---|
| Base Model | openai/whisper-medium |
| Parameters | ~769M |
| Language | Kreol Morisien (mfe) |
| Task | Transcription |
| Best WER | 12.82% |
Training Data
- Domain: In-domain Kreol Morisien speech
- Total Duration: ~15 hours
- Split: 90% train / 10% validation
- Format: 16kHz mono WAV
- Preprocessing: Text normalization (lowercased, whitespace-collapsed, smart quotes replaced)
Training Configuration
| Hyperparameter | Value |
|---|
| Epochs | 10 |
| Batch Size (per device) | 8 |
| Gradient Accumulation Steps | 4 |
| Effective Batch Size | 32 |
| Learning Rate | 3e-5 |
| LR Scheduler | Cosine |
| Warmup Steps | 200 |
| Weight Decay | 0.1 |
| Precision | bf16 |
| Generation Beams (eval) | 1 (greedy) |
| Best Checkpoint | Step 700 |
Training Results
| Step | Training Loss | Validation Loss | WER |
|---|
| 100 | 1.9289 | 0.3816 | 0.1964 |
| 200 | 0.4691 | 0.3208 | 0.1579 |
| 300 | 0.2571 | 0.3163 | 0.1499 |
| 400 | 0.1096 | 0.3442 | 0.1494 |
| 500 | 0.0398 | 0.3280 | 0.1360 |
| 600 | 0.0179 | 0.3312 | 0.1292 |
| 700 | 0.0046 | 0.3404 | 0.1282 |
| 750 | 0.0041 | 0.3405 | 0.1317 |
Usage
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import soundfile as sf
processor = WhisperProcessor.from_pretrained("Shagufta/whisper-medium-km-indomain10-withac")
model = WhisperForConditionalGeneration.from_pretrained("Shagufta/whisper-medium-km-indomain10-withac")
speech, sr = sf.read("audio.wav", dtype="float32")
input_features = processor.feature_extractor(
speech, sampling_rate=16000, return_tensors="pt"
).input_features
predicted_ids = model.generate(
input_features,
max_length=256,
num_beams=5,
)
transcription = processor.tokenizer.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)
Model Config
model.config.forced_decoder_ids = None
model.config.suppress_tokens = []
Framework
- Transformers (Seq2SeqTrainer)
- Evaluate (WER metric)