Voxtral Mini 4B Realtime Arabic
[!Note] See a space live demo directly in your browser here powered by this ONNX checkpoint. Many thanks to the ONNX community.
Voxtral Mini 4B Realtime Arabic is a streaming speech-to-text model for Arabic dialects and Modern Standard Arabic. It is designed to handle challenging real-world audio and code-switching, where speakers alternate between languages within the same conversation.
This model is fine-tuned from Voxtral-Mini-4B-Realtime-2602 and has approximately 4.4 billion parameters, with weights in BF16 precision. It processes 16 kHz audio through a causal audio encoder and language decoder, producing text as audio arrives. Its configurable transcription delay supports applications such as live captions and voice interfaces.
At a configured transcription delay of 480 ms, the model achieves an average Character Error Rate (CER) of 8.82% across seven Arabic benchmarks. It outperforms several open-source and proprietary offline models and comes within 0.91 percentage points of Voxtral Transcribe Arabic’s 7.91% CER.
The model is co-developed with the Ministère de la Transition Numérique et de la Réforme Administrative (MTNRA) under the umbrella of the strategic partnership signed between Mistral and Morocco in January 2026 as a sovereign AI asset.
Usage
vLLM
Use vLLM with Voxtral Realtime support and mistral-common audio dependencies.
pip install --upgrade vllm "mistral-common[audio]"
vllm serve mistralai/Voxtral-Mini-4B-Realtime-Arabic
Stream audio over WebSocket to /v1/realtime by adapting this example.
Transformers
Starting with transformers >= 5.2.0, Voxtral Realtime runs natively in Transformers.
See the Transformers documentation for details.
pip install --upgrade "transformers>=5.2.0" accelerate
pip install --upgrade "mistral-common[audio]"
import torch
from transformers import VoxtralRealtimeForConditionalGeneration, AutoProcessor
from mistral_common.tokens.tokenizers.audio import Audio
from huggingface_hub import hf_hub_download
repo_id = "mistralai/Voxtral-Mini-4B-Realtime-Arabic"
processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralRealtimeForConditionalGeneration.from_pretrained(
repo_id, dtype=torch.bfloat16, device_map="auto"
)
audio_file = hf_hub_download(
repo_id=repo_id,
filename="assets/bank-take-2.wav",
repo_type="model",
)
audio = Audio.from_file(audio_file, strict=False)
audio.resample(processor.feature_extractor.sampling_rate)
inputs = processor(audio.audio_array, return_tensors="pt")
inputs = inputs.to(model.device, dtype=model.dtype)
outputs = model.generate(**inputs, do_sample=False)
print(processor.batch_decode(outputs, skip_special_tokens=True)[0])
Benchmarks
We report Character Error Rate (CER) on established Arabic benchmarks and two new benchmarks, ISMA and Darija in the Wild, co-developed with Morocco’s Ministère de la Transition Numérique et de la Réforme Administrative (MTNRA). Lower CER is better.
All evaluations use temperature 0.0; the realtime model uses a configured transcription delay of 480 ms. CER is computed with normalization co-developed with MTNRA, covering Arabic dialect-specific spelling, clitics, loanwords, and spoken numbers. The code is available in normalization.py. Scores may differ under other normalization methods.
Example of usage:
from normalization import normalize_arabic_asr
reference = normalize_arabic_asr(reference)
hypothesis = normalize_arabic_asr(hypothesis)
Note: Gemini’s safety filters block transcription of some FLEURS audio samples.
Emerging capabilities
Training focused on Arabic code-switching. We also observe improvements in code-switching involving languages outside the training focus.
Limitations
The model targets Arabic transcription. For other languages, use the original Voxtral Mini 4B Realtime.
License
This model is licensed under the Apache 2.0 License.
You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.