arabart-xlsum-arabic
Fine-tuned moussaKam/AraBART for Arabic abstractive news summarization on the XL-Sum Arabic split.
Results
Evaluated on the full XL-Sum Arabic test set (4,688 article–summary pairs after cleaning) with the official XL-Sum scorer (csebuetnlp/xl-sum's multilingual_rouge_scoring with lang='arabic', the same Snowball-based stemmer used by the XL-Sum paper). Generation parameters match the paper: num_beams=4, no_repeat_ngram_size=2, length_penalty=0.6, max_length=84.
| Model | ROUGE-1 ↑ | ROUGE-2 ↑ | ROUGE-L ↑ | BLEU ↑ |
|---|---|---|---|---|
| AraBART (pretrained, no fine-tune) | 20.25 | 5.08 | 13.48 | 1.83 |
mT5-XLSum (zero-shot, csebuetnlp/mT5_multilingual_XLSum) | 34.82 | 14.77 | 29.17 | 7.43 |
| AraBART (fine-tuned, this model) | 34.99 | 15.77 | 29.56 | 8.26 |
Fine-tuning lifts AraBART by +14.7 ROUGE-1 / +10.7 ROUGE-2 / +16.1 ROUGE-L over its pretrained-only baseline, and matches/slightly beats the multilingual mT5-XLSum baseline on the Arabic split despite being a much smaller monolingual model.
Training data
XL-Sum (Hasan et al., 2021), a multilingual abstractive summarization dataset built from BBC news articles in 45 languages. We use the Arabic split (arabic_train.jsonl / arabic_val.jsonl / arabic_test.jsonl), distributed by the authors as arabic_XLSum_v2.0.tar.bz2.
Cleaning applied to every (document, summary) pair before training:
- Strip Arabic diacritics (
U+064B–U+065F,U+0670) and tatweel (U+0640). - Collapse whitespace.
- Drop pairs with <20 document words or <5 summary words.
Final cleaned counts: ~37k train / ~4.7k val / ~4.7k test.
Training procedure
Single Tesla T4 (16 GB) on Kaggle. Training framework: transformers Seq2SeqTrainer.
Hyperparameters
| Base model | moussaKam/AraBART (mBART arch., 6 enc / 6 dec layers, ~139 M params) |
| Train set | full cleaned XL-Sum Arabic train split (~37k pairs) |
| Eval set during training | 2,000 sampled pairs from the val split |
| Epochs | 3 |
| Per-device batch size | 4 |
| Gradient accumulation | 4 (effective batch 16) |
| Optimizer | AdamW (HF default) |
| Learning rate | 2e-5 |
| LR schedule | cosine, warmup ratio 0.1 |
| Weight decay | 0.01 |
| Label smoothing | 0.1 |
max_input_length | 512 |
max_target_length | 84 |
| Generation (eval) | num_beams=6, length_penalty=0.6 |
| Generation (inference / reported test) | num_beams=4, no_repeat_ngram_size=2, length_penalty=0.6, max_length=84 |
| Mixed precision | fp16 |
| Seed | 42 |
load_best_model_at_end=True with metric_for_best_model='eval_loss' — the published checkpoint is the epoch with the lowest validation loss.
Quick use
import re
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
REPO = "Omar10lfc/arabart-xlsum-arabic" # replace with your username if different
tok = AutoTokenizer.from_pretrained(REPO)
mdl = AutoModelForSeq2SeqLM.from_pretrained(REPO)
article = "..." # Arabic news article text
# Same minimal cleaning used during training (diacritics + tatweel + whitespace).
AR_DIACRITICS = re.compile(r"[ً-ٰٟ]")
TATWEEL = re.compile(r"ـ")
WS = re.compile(r"\s+")
def clean_arabic(t: str) -> str:
t = AR_DIACRITICS.sub("", t)
t = TATWEEL.sub("", t)
return WS.sub(" ", t).strip()
enc = tok(clean_arabic(article), max_length=512, truncation=True,
padding="max_length", return_tensors="pt")
ids = mdl.generate(
input_ids=enc["input_ids"],
attention_mask=enc["attention_mask"],
max_length=84,
min_length=10,
num_beams=4,
no_repeat_ngram_size=2,
length_penalty=0.6,
early_stopping=True,
)
print(tok.batch_decode(ids, skip_special_tokens=True,
clean_up_tokenization_spaces=False)[0])
Or with the HF pipeline:
from transformers import pipeline
summ = pipeline("summarization", model=REPO, tokenizer=REPO)
print(summ(article, max_length=84, min_length=10, num_beams=4,
no_repeat_ngram_size=2, length_penalty=0.6)[0]["summary_text"])
Intended use & limitations
- Intended use: Modern Standard Arabic (MSA) abstractive summarization of news-style articles. Used as the summarization stage of an Arabic audio understanding pipeline (Whisper ASR → AraBART summarization → CAMeL-BERT/FAISS semantic search).
- Out of scope: dialectal Arabic (Egyptian, Levantine, Maghrebi, Khaleeji), social-media text, code-switching with English, very long inputs (>512 BPE tokens are truncated), domains far from BBC news (legal, scientific, conversational).
- Bias: XL-Sum is built from BBC news, so the model inherits BBC's editorial perspective and topical distribution. Summaries may reflect those biases, particularly on geopolitical topics. As with all abstractive models, generated summaries may hallucinate facts not present in the source — verify before downstream use.
Citation
@misc{arabart-xlsum-arabic,
title = {arabart-xlsum-arabic: Fine-tuned AraBART for Arabic abstractive summarization on XL-Sum},
author = {{Omar10lfc}},
year = {2026},
howpublished = {Hugging Face},
note = {Fine-tune of moussaKam/AraBART on the Arabic split of XL-Sum, ROUGE-1 34.99 / ROUGE-2 15.77 / ROUGE-L 29.56.}
}
Built on top of:
@inproceedings{kamal-eddine-etal-2022-arabart,
title = {{A}ra{BART}: a Pretrained {A}rabic Sequence-to-Sequence Model for Abstractive Summarization},
author = {Kamal Eddine, Moussa and Tomeh, Nadi and Habash, Nizar and
Le Roux, Joseph and Vazirgiannis, Michalis},
booktitle = {Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP)},
year = {2022}
}
@inproceedings{hasan-etal-2021-xl,
title = {{XL}-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages},
author = {Hasan, Tahmid and Bhattacharjee, Abhik and Islam, Md. Saiful and
Mubasshir, Kazi and Li, Yuan-Fang and Kang, Yong-Bin and
Rahman, M. Sohel and Shahriyar, Rifat},
booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
year = {2021}
}