FarukSTT — Tunisian Derja Speech-to-Text
FarukSTT is the best publicly available ASR model for Tunisian Arabic (Derja), fine-tuned from Whisper Large v3. It handles real-world Tunisian speech including natural code-switching between Arabic, French, and English.
Whisper Large v3 out of the box scores above 50% WER on Tunisian Derja. This model brings that down to ~32% through targeted fine-tuning on real Tunisian conversational data.
Performance
| Model | WER |
|---|---|
| Whisper Large v3 (baseline) | >50% |
| FarukSTT v1 | 33.26% |
| FarukSTT v2 | 31.99% |
Evaluated on FARUKxAUTO/tunisian-asr-cleaned validation split.
Key Features
- Tunisian Derja — not MSA, real dialectal Arabic
- Code-switching — Arabic + French + English mid-sentence
- Real-world speech — podcasts, interviews, conversations
- 54k training samples — 12.7GB of audio data
Intended Use
- Tunisian Arabic transcription
- Meeting and interview transcription for Tunisian speakers
- Input for downstream NLP tasks in Derja
Limitations
- Optimized for Tunisian dialect, not Modern Standard Arabic
- WER ~32% — suitable for assisted transcription, not verbatim accuracy
- May struggle in very noisy environments
Training Data
Fine-tuned on FARUKxAUTO/tunisian-asr-cleaned — 54,156 audio-transcription pairs of real Tunisian speech including natural code-switching.
Training Procedure
v2 Hyperparameters
- learning_rate: 1e-6
- train_batch_size: 2
- gradient_accumulation_steps: 16 (effective batch: 32)
- warmup_steps: 200
- fp16: True
- optimizer: AdamW
- Framework: Transformers 4.45.0, PyTorch 2.11.0
Training Results (v1)
| Step | Validation Loss | WER |
|---|---|---|
| 500 | 0.3228 | 34.02% |
| 2000 | 0.3121 | 32.79% |
| 4000 | 0.3110 | 33.26% |
Training Results (v2)
| Step | Validation Loss | WER |
|---|---|---|
| 250 | 0.3114 | 32.90% |
| 500 | 0.3060 | 31.99% |
Usage
from transformers import pipeline
pipe = pipeline("automatic-speech-recognition", model="FARUKxAUTO/FarukSTT")
result = pipe("audio.wav")
print(result["text"])
Citation
If you use this model, please credit:
FarukSTT by FARUK BATTIKH — Tunisian Derja ASR, 2024