Rawi

Rawi is a Classical Arabic encoder adapted to pre-modern biographical and historical prose, with a focus on the Islamic West (the Maghrib and al-Andalus). It is built for token- and span-level work on these texts: names, nisbas, places, offices and chains of transmission. On held-out western biographical entries it matches CAMeLBERT-CA with an 8,192-token context, and halves the pseudo-perplexity of the model it was adapted from. Rāwī is Arabic for the transmitter in an isnād.

Model details

• Developed by: Abderahmane Ainouche and Riadh Moulahcene, École Nationale Supérieure d'Intelligence Artificielle (ENSIA), Algiers, as part of the SILA project
• Model type: ModernBERT encoder, masked language model, 149,587,560 parameters
• Language: Classical Arabic
• Base model: NAMAA-Space/AraModernBert-Base-V1.0 at revision 5b76dac145f7d4b71844c685c0627df6fc5e6fd9
• Context length: 8,192 tokens
• Tokenizer: the base model's, unchanged (50,280 entries)
• Licence: CC BY-NC-SA 4.0

How to use

Rawi needs transformers 5 (tested on 5.16.1) and no custom code. FlashAttention-2 is optional.

import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

repo = "Al-Sila/Rawi"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForMaskedLM.from_pretrained(repo).eval()

text = f"حدثنا محمد {tokenizer.mask_token} عبد الله"
inputs = tokenizer(text, return_tensors="pt")
position = (inputs["input_ids"][0] == tokenizer.mask_token_id).nonzero().item()
with torch.inference_mode():
    logits = model(**inputs).logits[0, position]
print([tokenizer.decode([int(i)]).strip() for i in logits.topk(5).indices])
# ['بن', 'حدثنا', 'أبو', 'ثنا', 'قال']

For entity recognition, onomastic parsing or relation extraction, load it with AutoModelForTokenClassification or as the encoder of your own head, and fine-tune.

Uses

Intended use. Research on pre-modern Arabic texts: a starting encoder for named-entity recognition, onomastic parsing, relation extraction and retrieval over biographical dictionaries, chronicles and geographical works, and fill-mask over historical prose.

Out of scope. Modern Standard Arabic and dialectal Arabic, which it was not adapted to. Commercial use, which the licence does not allow.

Training data

The adapt selection of OpenITI release 2025.1.9: 5,620 texts and 985,096,880 tokens of pre-modern Arabic prose. Texts flagged in the release as uncorrected OCR are excluded. Duplicate segments are removed. Text is normalised to the base tokenizer's orthography: presentation forms, zero-width characters, no-break spaces and tatwil.

The mixture is weighted towards the Islamic West. Western means written by an author attributed to the Maghrib or al-Andalus.

Held out from training:

• a fixed random 20% of the entries of the western biographical dictionaries used for annotation
• three books in full: Ibn ʿAbd al-Barr's al-Istīʿāb, Ibn Farḥūn's al-Dībāj al-mudhahhab and Ibn ʿAbd al-Malik al-Marrākushī's al-Dhayl wa-l-takmila
• evaluation segments of western, eastern and eastern biographical text

Training procedure

Evaluation

The base model and Rawi are scored with the same instruments on held-out text. Loss is masked language-model cross-entropy per token at a fixed 15% mask. Intervals are 95%, from 10,000 bootstrap resamples over books.

Pseudo-perplexity per word on 200 held-out western biographical entries of any length, read whole (lower is better):

Held-out loss by population (lower is better):

Loss falls in every population, including the three books held out in full.

Onomastic cloze. The nisba or place of origin (min ahl …) in held-out western entries is masked and predicted, top-1 accuracy:

Comparison with other Arabic encoders

Every model reads the same held-out western biographical entries: 200 drawn across 32 books, at most 7 from any one, 20 to 130 words each, with hamza, tāʾ marbūṭa and alif maqṣūra kept as written. The 3 entries that exceed one model's 256-token scoring window are set aside for all of them, leaving 197. Tokens per word is measured on 40 western texts. Intervals are 95%, from 10,000 bootstrap resamples over entries. The difference column is paired on the same entries (lower is better).

Rawi halves its base model's pseudo-perplexity and matches CAMeLBERT-CA, the established Classical Arabic encoder, with sixteen times its context. CAMeLBERT-CA's classical training corpus is an earlier OpenITI release, so it may have read these entries. Rawi's adaptation corpus excludes them.

What was not measured

• Downstream tasks. Fine-tuned results for entity recognition, onomastic parsing and relation extraction will be reported when the annotated gold data is complete.
• Behaviour on Modern Standard Arabic, dialectal Arabic and manuscript transcriptions.

Limitations

• Western text is still harder for the model than eastern text. The western minus eastern loss gap is 0.275 for the base model and 0.282 for Rawi.
• Some western nisbas are weak. In fill-mask, al-Maghribī and al-Ifrīqī still rank below common eastern nisbas such as al-Baṣrī.
• The source texts are printed critical editions, mostly unvocalised. Input with full diacritics is outside what the model was trained on.
• Results are from one training run.

Glossary

• Isnād: the chain of transmitters through whom a report or a text was received, written as a sequence such as ḥaddathanā X ʿan Y.
• Nasab: the line of descent in a name, joined by ibn (son of), as in Muḥammad ibn ʿAbd Allāh.
• Kunya: a name formed with Abū or Umm (father or mother of), as in Abū Bakr.
• Nisba: an adjective of origin, school, tribe or trade, as in al-Andalusī, al-Mālikī or al-Qurṭubī.
• Min ahl …: "from the people of …", the formula that gives a scholar's town or region in a biographical entry.
• Pseudo-perplexity: a masked language model's perplexity computed by masking each token in turn and scoring the model's prediction for it. Per word, so models with different tokenizers are comparable.

Licence

CC BY-NC-SA 4.0. The base model is Apache-2.0. The training corpus, OpenITI, is CC BY-NC-SA 4.0, and its share-alike terms are applied to the weights. Models built on Rawi carry the same licence.

Citation

@misc{sila_rawi_2026,
  title  = {{Rawi}: A Classical Arabic Encoder for the Biographical Literature of the Islamic West},
  author = {Ainouche, Abderahmane and Moulahcene, Riadh},
  year   = {2026},
  url    = {https://huggingface.co/Al-Sila/Rawi},
  note   = {Base model NAMAA-Space/AraModernBert-Base-V1.0; trained on OpenITI 2025.1.9 (CC BY-NC-SA 4.0)}
}