Tashkeel-v2 (تشكيل)
A state-of-the-art Morpheme-Aware Arabic Diacritization (تشكيل) Model powered by UBC-NLP/MARBERTv2 with a Dual-Head Multi-Task Architecture (Stem Morphology + 3x times Weighted I'rab Case Endings).
Trained on a curated, deduplicated, contamination-free multi-corpus of 5,006,013 Classical & Modern Arabic sentences and fine-tuned on Misraj/Sadeed_Tashkeela.
Tashkeel-v2 achieves 2.85% Total DER and 0.00% Hallucinations on the standard Misraj/SadeedDiac-25 benchmark, outperforming GPT-4 (3.86% DER) and Gemini-Flash-2.0 (3.19% DER) while processing ~898 sentences per second on a single GPU.
Benchmark Evaluation on SadeedDiac-25 (1,200 Sentences)
Evaluated with standard Arabic Morph/Total DER & WER:
- Total DER (CE): Diacritic Error Rate including case endings (I'rab).
- Morph DER (w/o CE): Diacritic Error Rate on internal stem vowels only.
- Hallucinations: Share of examples rejected due to character or word-count mismatch.
| Model / System | Total Scored | Hallucination Rate | Total DER (CE) | Morph DER (w/o CE) | Total WER (CE) | Morph WER (w/o CE) | Inference Speed |
|---|---|---|---|---|---|---|---|
| Claude-3.7-Sonnet | 1,190 / 1,200 | 0.82% | 1.39% | 0.77% | 4.67% | 2.31% | ~5 sent/s |
| Tashkeel-v2 (This Model) | 1,200 / 1,200 | 0.00% | 2.85% | 2.13% | 8.02% | 5.66% | ~898 sent/s |
| Tashkeel-50M (v1) | 12 / 1,200* | ~99.0%* | 3.08%* | 2.26%* | 9.56%* | 6.77%* | ~25 sent/s |
| Gemini-Flash-2.0 | 1,186 / 1,200 | 1.17% | 3.19% | 2.38% | 7.99% | 5.50% | ~20 sent/s |
| GPT-4 | 1,188 / 1,200 | 1.02% | 3.86% | 3.86% | 5.27% | 10.93% | ~10 sent/s |
| Sadeed (Published Paper) | 1,114 / 1,200 | 7.19% | 7.29% | 5.26% | 13.74% | 9.92% | ~35 sent/s |
| tash2 (50M Baseline) | 1,199 / 1,200 | 0.08% | 15.29% | 11.41% | 24.36% | 20.40% | ~25 sent/s |
*Starred models are scored only on the subset of length-matched outputs.
Key Innovations in Tashkeel-v2
- Zero Hallucinations by Design (1:1 Character Sequence Tagging): Unlike generative causal LMs that drop leading numbers, brackets, or repeat words, Tashkeel-v2 operates as a character-level sequence tagger. The input characters, numbers, and punctuation are 100% immutable.
- Morpheme & Clitic Segmentation Engine:
Decomposes Arabic words into
[PROCLITIC](e.g.فـ,بـ,الـ),[STEM], and[ENCLITIC](e.g.ـهم,ـها) character spans. This explicitly marks the true morphological stem ending where I'rab vowels belong. - Dual-Head Multi-Task Loss: Features a dedicated I'rab Head weighted at 3 times on stem endings and word boundaries, giving strong gradient signal to resolve long-range grammatical case assignments.
- 5.0 Million Deduplicated Pretraining: Pretrained on 5,006,013 unique sentences from Sadeed, Shamela, Ashaar, and Quran with 100% hash deduplication and zero contamination with the evaluation set.
Model Details
- Model Type: Character-Level Arabic Diacritizer
- Backbone:
UBC-NLP/MARBERTv2(12 Layers, 768-d hidden, 100k vocabulary) - Parameters: 171.57 Million (100% trainable)
- Hidden Layers: 12 Transformer Layers + 2-layer Bidirectional LSTM (768-d) + Dual Multi-Task Heads
- Precision:
bfloat16/float32 - Supported Diacritic Classes: 15 classes (None, Fatha, Damma, Kasra, Sukun, Tanween Fath/Damm/Kasr, Shadda, Shadda+Vowels)
Quick Start
Installation
pip install torch transformers pyarabic safetensors
Python Usage
import torch
from transformers import AutoTokenizer
from huggingface_hub import hf_hub_download
# 1. Download model file
repo_id = "ahmedsamirtarjama/Tashkeel-v2"
device = "cuda" if torch.cuda.is_available() else "cpu"
# Download modeling code
script_path = hf_hub_download(repo_id=repo_id, filename="modeling_tashkeel.py")
weights_path = hf_hub_download(repo_id=repo_id, filename="model.safetensors")
# Load model class
import importlib.util
spec = importlib.util.spec_from_file_location("modeling_tashkeel", script_path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = mod.MorphemeAwareArabicDiacritizer().to(device)
from safetensors.torch import load_file
model.load_state_dict(load_file(weights_path))
model.eval()
# 2. Run Diacritization
text = "اللغة العربية لغة جميلة وثرية بالمفردات"
result = model.diacritize(text, tokenizer=tokenizer, device=device)
print(f"Input : {text}")
print(f"Output: {result}")
# Output: اللُّغَةُ الْعَرَبِيَّةُ لُغَةٌ جَمِيلَةٌ وَثَرِيَّةٌ بِالْمُفْرَدَاتِ
Batch Diacritization
sentences = [
"فبمستحقاتهم ولأطأنها",
"حدثني يحيى عن مالك عن هشام بن عروة عن أبيه (2 / 192)",
"قوله (وإن جعلا مدة قد تكمل وقد لا تكمل، فهل تصح؟ على وجهين) .",
"- ولو أن أمير عسكر المسلمين أهدى إلى ملك العدو فعوضه"
]
results = model.diacritize(sentences, tokenizer=tokenizer, device=device, batch_size=64)
for inp, out in zip(sentences, results):
print(f"{inp} -> {out}")
Training Recipe
| Hyperparameter | Pretraining Phase | Target Calibration Phase |
|---|---|---|
| Dataset | 5,006,013 Sentences (Sadeed + Shamela + Ashaar + Quran) | 1,007,080 Sentences (Sadeed_Tashkeela) |
| Encoder LR | 2*10^-5 | 8*10^-6 |
| Heads / LSTM LR | 3*10-4 | 1.2*10^-4 |
| LR Scheduler | Cosine with 3% Warmup | Cosine Decay down to 1*10^-7 |
| Effective Batch Size | 128 (DDP across 2x NVIDIA L40S) | 128 (DDP across 2x NVIDIA L40S) |
| I'rab Loss Weight | 3 times | 3 times |
| Label Smoothing | 0.03 | 0.03 |
| Mixed Precision | bfloat16 | bfloat16 |
Intended Use & Capabilities
- Full Text Diacritization: Classical Arabic (Fusha), legal texts, literature, and Islamic heritage books.
- Text-to-Speech (TTS) Front-End: Providing accurate vowel and case ending tokens for Arabic TTS synthesis.
- High-Throughput Production APIs: Capable of processing over 600–900 sentences per second per GPU.
- Search & NLP Normalization: Exact pronunciation disambiguation.
Citation
If you use Tashkeel-v2 in your work, please cite:
@misc{tashkeelv2_2026,
title = {Tashkeel-v2: Morpheme-Aware Dual-Head Arabic Diacritization},
author = {Ahmed Samir},
year = {2026},
howpublished = {\url{https://huggingface.co/ahmedsamirtarjama/Tashkeel-v2}}
}