alexmt-qwen3-0.6b-simpo-bwchrf
Context-aware English ↔ dialectal Arabic translation for conversational turns, trained on Alexandria.
SimPO (beta 2.0, gamma 1.0, lr 2e-6, 3 epochs) on top of alexmt-qwen3-0.6b-sft.
Preference pairs. The SFT model generated 5 candidates (1 greedy + 4 sampled) for each source turn of a train fold held out from SFT. Candidates were ranked by sentence chrF++ against the human reference; chosen = best, rejected = worst (both are model outputs; the reference is only the yardstick).
Data. Alexandria train split, 9 varieties (EG, JO, LB, MR, OM, PS, SA, SY, YE), both directions. The train split is divided into 5 conversation-level folds; SFT uses folds 1-4, and fold 0's source turns feed the preference pairs. Each example includes up to 3 prior turns as context, the domain, the participants, and the speaker/addressee gender direction.
Results
Alexandria public test split, 9 varieties, macro-average over varieties; greedy decoding, max 128 new tokens, our own prompt/decoding setup (not the Alexandria paper's).
| model | en→dialect spBLEU | chrF++ | len/ref | dialect→en spBLEU | chrF++ | len/ref |
|---|---|---|---|---|---|---|
| alexmt-qwen3-0.6b-sft | 9.17 | 25.70 | 0.99 | 24.49 | 44.18 | 0.97 |
| alexmt-qwen3-0.6b-dpo-bwchrf-lr2e6 | 8.84 | 27.17 | 1.15 | 23.78 | 46.47 | 1.12 |
| alexmt-qwen3-0.6b-dpo-bwspbleu-lr2e6 | 9.65 | 26.41 | 0.99 | 25.81 | 45.72 | 0.96 |
| alexmt-qwen3-0.6b-simpo-bwchrf (this model) | 8.61 | 27.35 | 1.19 | 21.70 | 46.23 | 1.21 |
| alexmt-qwen3-0.6b-simpo-bwspbleu | 9.72 | 26.60 | 1.00 | 25.85 | 45.81 | 0.98 |
Please read these before comparing: single training seed; one data fold; the DPO learning rate was compared on the test split itself (a held-out-dev selection is in progress), so small differences between the DPO rows are not established. Scores come from this setup and are not directly comparable to the Alexandria paper's tables (zero-shot models, different decoding) or to the AlexandriaX shared task (a different, private test set). Only overlap metrics (spBLEU, chrF++) have been computed; dialect fidelity has not been evaluated.
Usage
The model expects Alexandria's JSON prompt as a single user turn and answers with
{"translation": "..."}. Apply the tokenizer's chat template with add_generation_prompt=True.
Translate the given turn of a conversation from English to Egyptian Arabic (Cairene) Dialect, considering the previous context if provided.
Input: {"country": "EG", "domain": "Agriculture and farming", "participants": ["Waterresourcemanager", "Farmer"], "context": [], "current_turn": {"speaker": "Waterresourcemanager", "gender_direction": "male -> male", "text": "Good morning. Is the water reaching your land properly according to the schedule?"}}
Guidelines:
- Return the result strictly in valid JSON.
- Translate to Egyptian Arabic (Cairene) Dialect using Arabic script.
- Do not add any code, explanations, comments, or any other extra text.
- Keep the meaning and tone and respect the gender direction.
- Consider the country, the domain, the participants, and the speaker in your translation.
- Only translate the "text" field of the "current_turn".
- If a context is provided, do not translate it, and use it to inform your translation.
Output scheme: { "translation": "translation of the text from the current turn" }
Licence
CC BY-NC 4.0 (non-commercial), because the training data (Alexandria) is CC BY-NC 4.0. The base model, Qwen3-0.6B is Apache-2.0.
Citation
If you use this model, please cite the Alexandria dataset:
@inproceedings{elmekki2026alexandria,
title = {Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs},
author = {El Mekki, Abdellah and others},
booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL)},
year = {2026},
url = {https://aclanthology.org/2026.acl-long.1503/}
}