Tashkeel-v4: Arabic Diacritization (MARBERTv2 + BiLSTM + Linear-Chain CRF)

Research use only. This model is released under a non-commercial, research-only license. Commercial or business use is not permitted. See License.

Tashkeel-v4 adds full diacritics (تشكيل) to Arabic text: Classical and Modern Standard Arabic, including case endings. It is a character tagger: it predicts one diacritic class per input character. So it never adds, drops or changes letters, digits or punctuation (0% hallucination by construction), and it's fast enough for batch processing.

Results

All numbers use the official SadeedDiac-25 evaluator (ArabicDiacritizationEvaluator, gt_missing_diacritic_is_error=False). Total = including case endings; Morph = excluding the last letter. Every sentence is scored (no hallucinations).

These scores include the built-in إ post-processing step (see below). The package was verified to reproduce them exactly: its predictions match the evaluated checkpoint on every benchmark sentence.

Compared with Tashkeel-v3: SadeedDiac-25 DER 2.26 → 1.4 (−31%), CATT DER 4.98 → 2.95 (−41%).

Throughput (one NVIDIA L40S, diacritize(..., batch_size=64), including Python preprocessing): ~34 sentences/s on SadeedDiac-25 (long, multi-clause sentences) and ~111 sentences/s on CATT (shorter news sentences).

Generative LLMs are scored only on the sentences where their output kept the input letters intact. Tashkeel-v4 is close to the strongest LLM result while being far cheaper and faster, and it never alters the input text.

Comparison with the models in the CATT paper

The CATT paper (Alasmary et al., 2024) compares 13 systems on its benchmark. We ran Tashkeel-v4 on the paper's released input file and scored it with the authors' own evaluation script (compute_der.py from abjadai/catt), next to the model outputs the authors released. We checked that the script reproduces the paper's published numbers exactly (e.g. CATT ED: 8.624 / 34.191).

This protocol scores every letter, including letters the reference deliberately leaves unmarked. That's why all numbers here are higher than the CATT row in the Results table above, which uses the SadeedDiac-25 evaluator. CE = with case endings.

Tashkeel-v4 has the lowest error on all four metrics, with and without case endings. † The CATT script flags sentences whose output text changed (GPT-4: 11, Command R+: 77, Sakhr: 53, Mishkal: 1); Tashkeel-v4: 0. Our training data was checked against the CATT benchmark for 10-word overlap (see Training).

Usage

Access is gated: accept the conditions on this page (logged in to Hugging Face), then authenticate before downloading.

pip install torch transformers
huggingface-cli login   # or set the HF_TOKEN environment variable

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "ahmedsamirtarjama/Tashkeel-v4"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device).eval()

print(model.diacritize("ذهب الطالب المجتهد إلى المدرسة صباحا", tokenizer=tokenizer, device=device))

texts = ["قل إن صلاتي ونسكي ومحياي ومماتي لله رب العالمين", "سطا لصوص على منزل وزيرة الدفاع التشيلية"]
print(model.diacritize(texts, tokenizer=tokenizer, device=device, batch_size=64))

Input should be undiacritized Arabic text. Characters that aren't Arabic letters pass through unchanged.

Model

• Encoder: UBC-NLP/MARBERTv2 (12 layers, 768-d). Its token vectors are mapped onto characters.
• Character features: character, position-in-word and clitic-segment (proclitic / stem / enclitic) embeddings.
• Sequence layers: 2-layer BiLSTM with a residual projection. Two heads: one for word-internal diacritics, one for case endings on word-final letters.
• Decoding: linear-chain CRF over 15 diacritic classes (none, fatha, damma, kasra, sukun, the three tanweens, shadda, and shadda combinations). Globally optimal decoding with Viterbi.
• Post-processing: a bare hamza-below alef (إ) always receives its kasra. This is linguistically always correct, and no other character is modified.
• Size: ~171M parameters (fp32 weights). Same architecture and code as Tashkeel-v3; only the weights changed.

Training

Tashkeel-v4 continues from Tashkeel-v3 and was trained in stages. In every stage each candidate was compared with a matched control trained for the same number of steps; the final checkpoint of each of 2 seeds was kept and the two were weight-averaged. This rule was fixed in advance, and no checkpoint was ever selected on the benchmark test sets.

1. In-domain base: Misraj/Sadeed_Tashkeela, replayed in every stage.
2. Complete classical text: sentences from the Tashkeela corpus (v0.3), Shamela and other public diacritized collections, kept only if fully diacritized (every word-final letter marked, ≤2% unmarked interior letters). Sukun before hamzat al-wasl was normalized to the connecting-vowel convention used by Sadeed_Tashkeela (مِنَ الـ، عَنِ الـ).
3. Privately collected annotated data: ~114k news and classical sentences, privately collected and diacritized with our in-house annotation pipeline. Outputs were kept only if the letters were unchanged, every word-final letter was marked, and the text passed the same normalization as above.
4. Model-assisted news labeling: news sentences from arbml/SANAD labeled by earlier versions of the model, kept only where an independent model agreed on ≥97% of words (several rounds, each relabeled by the latest model).
5. Wikipedia: ~29k Arabic Wikipedia chunks whose own diacritics closely matched the model's.

The continued-training data (stages 2–5, ~1M sentences) is published as ahmedsamirtarjama/Tashkeel-data under the same research-only terms.

Decontamination: every added sentence was checked against SadeedDiac-25, the CATT benchmark and our development set, and any sentence sharing a 10-word sequence with any of them was removed.

Limitations

• Rare and foreign words are the main remaining error source. Words seen 50 times or fewer in training cause about 33% (SadeedDiac-25) and 37% (CATT) of the errors. That includes transliterated names and brands and uncommon classical vocabulary.
• Case endings still depend on syntax the model sometimes misreads, for example a fronted object vs. the subject.
• Conventions: the model writes the connecting vowel before hamzat al-wasl (مِنَ الْ), as in its training data. The SadeedDiac-25 references are inconsistent here (e.g. من is written with sukun in ~30% of wasl contexts, أي and لو in ~95%), which costs ≈0.18 DER. The CATT references also mark the wasl alef itself (اِحْتَفَلَ), which this model doesn't.
• Scoring note: the official evaluator drops a shadda written after its vowel in a reference, so a few correct predictions count as errors on SadeedDiac-25 (≈0.05 DER).
• Dialectal Arabic and poetry-specific diacritization weren't targets of this model.

License

Research use only. Tashkeel-v4 (weights and code) is released under the Tashkeel Research-Only License (see LICENSE): use, copying and modification are allowed only for non-commercial research and educational purposes. Commercial or business use of any kind is not permitted. Publications using the model should cite this repository.