Turath-OCR-Fix
Turath-OCR-Fix repairs the damage OCR does to Arabic and Persian text. It restores the hamzas an engine flattened (الاسلام → الإسلام), puts back the dots it lost from پ چ ژ گ (کفت → گفت), undoes the letter substitutions that turn a word into a non-word (حيات ه → حياته), strips tatweel and orphaned diacritics, fixes comma spacing, and re-normalizes Persian letters that drifted into Arabic forms and back.
It is a character-level model — google/byt5-base (582M parameters), byte vocabulary — so it sees the letters themselves rather than subword tokens, which is what a task like this needs. Built by Sadiqoon Technologies.
نموذج يُصلح ما يفسده المسح الضوئيّ في النصوص العربية والفارسية: يردّ الهمزات الساقطة، ويعيد النقاط المطموسة من پ چ ژ گ، ويصحّح الحروف الملتبسة، ويحذف التطويل والتشكيل اليتيم، ويضبط المسافات وعلامات الترقيم، ويعيد الحروف الفارسية إلى رسمها. نموذج على مستوى الحرف (ByT5) لأن المهمة مهمة حروف لا كلمات.
Evaluation
600 held-out passages, character error rate against the clean original:
| CER | In plain terms | |
|---|---|---|
| OCR-damaged input | 2.65% | one wrong character every ~38 — roughly 26 errors on a printed page |
| After Turath-OCR-Fix | 0.74% | one wrong character every ~135 — roughly 7 errors on a page |
72% of the character errors removed. On a 300-page book that is the difference between some 7,800 corrupted characters and some 2,200: a text a proofreader can finish, instead of one they would rather retype.
The model is conservative by design: it corrects what it recognizes and leaves the rest untouched rather than rewriting the passage. In heritage texts a residual error is far cheaper than an invented one — a wrong letter is visible to any reader, a fluent guess is not.
Training
98,000 passage pairs built from a large Arabic and Persian encyclopedic corpus with sound orthography. Each clean passage was paired with a corrupted version produced by an OCR-noise model: letter confusions drawn from real shape-similarity groups (ب/ت/ن/ي/ث, ج/ح/خ, د/ذ, ص/ض, ع/غ…), hamza flattening, Persian dot loss, Arabic↔Persian letter drift, tatweel insertion, ZWNJ loss, character drops and duplications, word merges and splits, and punctuation damage — applied at four intensity levels from 8% to 35% of words. One epoch.
Usage
from transformers import AutoTokenizer, T5ForConditionalGeneration
tok = AutoTokenizer.from_pretrained("sadiqoon/turath-ocr-fix")
model = T5ForConditionalGeneration.from_pretrained("sadiqoon/turath-ocr-fix").eval()
def fix(text):
t = tok(text, return_tensors="pt", truncation=True, max_length=384)
return tok.decode(model.generate(**t, max_new_tokens=384)[0], skip_special_tokens=True)
fix("الاسلام دیـن الفطرة,وقد جاء بما يوافق العقل فى كل شى")
Feed it a sentence or a short passage at a time — up to ~380 bytes works best. For a whole book, split on sentence boundaries and process in batches.
Citation
@misc{sadiqoon2026turathocrfix,
title = {Turath-OCR-Fix: Character-Level Post-Correction for Arabic and Persian OCR},
author = {Sadiqoon Technologies},
year = {2026},
url = {https://huggingface.co/sadiqoon/turath-ocr-fix}
}
License & Contact
MIT. Built and maintained by Sadiqoon Technologies Ltd, London. Questions and collaboration: info@sadiqoon.uk