Whisper Large V3 Turbo — Egyptian Arabic Code-Switching

Fine-tuned openai/whisper-large-v3-turbo for Egyptian Arabic with Arabic↔English code-switching support.

Model Details

What Does It Do?

The base Whisper model transcribes everything in Arabic script, even English words. This fine-tuned version outputs English words in English when the speaker code-switches.

Example

Audio content: A speaker saying "تعالوا نبني مع بعض برنامج توصيل أكل... من غير ما نكتب ولا سطر code... عن طريق منصة Replit... منزلين عندهم Agent 4"

Usage

Using pipeline (simplest)

from transformers import pipeline, AutoProcessor
import torch

model_id = "mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching"

# Manually load the processor
processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model_id,
    processor=processor,
    torch_dtype=torch.float16, # Better for CUDA
    device="cuda",
    chunk_length_s=30,
)

result = pipe(
    "/path/to/audio.wav", 
    generate_kwargs={"language": "arabic", "task": "transcribe"},
)

print(result["text"])

Using Processor + Model

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

model_id = "mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching"

processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.bfloat16).to("cuda")

# Load audio
audio, sr = librosa.load("path/to/audio.wav", sr=16000)

# Process
input_features = processor.feature_extractor(
    audio, sampling_rate=16000, return_tensors="pt"
).input_features.to(device="cuda", dtype=torch.bfloat16)

# Generate
with torch.no_grad():
    generated_ids = model.generate(
        input_features,
        max_new_tokens=444,
        language="ar",
        task="transcribe",
    )

# Decode
text = processor.tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)

Batch Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

model_id = "mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching"
processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.bfloat16).to("cuda")

audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
audios = [librosa.load(f, sr=16000)[0] for f in audio_files]

inputs = processor.feature_extractor(
    audios, sampling_rate=16000, return_tensors="pt", padding=True
)
input_features = inputs.input_features.to(device="cuda", dtype=torch.bfloat16)

with torch.no_grad():
    generated_ids = model.generate(
        input_features,
        max_new_tokens=444,
        language="ar",
        task="transcribe",
    )

texts = processor.tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
for t in texts:
    print(t)

Training Details

• Method: Encoder frozen, decoder-only fine-tuning
• Dataset: 12,180 train / 300 eval samples from MohamedRashad/arabic-english-code-switching
• Learning rate: 1e-5 with cosine schedule
• Batch size: 16 (8 × 2 gradient accumulation)
• Epochs: 5
• Precision: bfloat16

Limitations

• Optimized for Egyptian Arabic dialect — may not generalize well to other Arabic dialects
• Code-switching performance is best for common English terms
• Audio should be 16kHz mono for best results
• Maximum audio length: 30 seconds per segment

Citation

If you use this model, please cite the base model and dataset:

@misc{whisper-large-v3-turbo-egyptian-cs,
  title={Whisper Large V3 Turbo — Egyptian Arabic Code-Switching},
  author={Mohammed Aly},
  year={2025},
  url={https://huggingface.co/mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching}
}