Pira (پیرا) - Abstractive Persian Text Summarization
Pira is a lightweight, highly efficient abstractive text summarization model specifically fine-tuned for the Persian language.
The Meaning Behind the Name
Pira (پیرا) is derived from the Persian infinitive Pirastan (پیراستن), which means to prune, trim, or purify. Just as a gardener prunes a tree to reveal its core structure and encourage healthy growth, this model is designed to strip away verbose journalistic fluff, rhetorical padding, and excess details to extract the pure, core message of a Persian text.
Model Details
- Model Type: Sequence-to-Sequence (Seq2Seq)
- Base Architecture:
google/mt5-small(~300M parameters) - Language: Persian / Farsi (
fa) - Task: Abstractive Text Summarization
- Developer: Mahdi Alikhah
Training Data
To prevent the model from learning lazy "extractive" habits (simply copying the first paragraph), Pira was trained on a meticulously blended and balanced corpus of ~96,000 curated rows:
csebuetnlp/xlsum(Persian Split): ~47k rows of BBC Persian articles. This provided the foundation for high-quality, professional, and purely abstractive journalistic summaries.RohanAiLab/persian_daily_news(Sampled): 50k rows downsampled from a massive 2M-row dataset of domestic Iranian news (ISNA, Tasnim, etc.) to ensure deep coverage of local political, economic, and social vocabulary without overpowering the abstractive traits of XL-Sum.msdos34/persian-summary-dataset: ~1k rows formatted as Instruction-Tuning data, helping the model learn prompt responsiveness.
Training Environment & Hardware
The model was fine-tuned to completion with optimal efficiency using modern hardware acceleration.
- Hardware: Single NVIDIA L4 GPU (Google Colab environment)
- Compute Precision:
bfloat16(bf16) native mixed precision, preventing NaN gradient explosions and significantly accelerating training time. - Epochs: 3
- Total Training Steps: 18,066
- Batch Size: 16 (per device)
- Learning Rate: 5e-4
- Optimizer: AdamW
Evaluation Results
The training run demonstrated textbook convergence with zero signs of overfitting:
- Final Training Loss:
1.895 - Final Validation Loss:
1.758
Note on Loss: A validation loss in the 1.7 range for generative tasks indicates strong abstractive capabilities. Extractive models often achieve lower loss by memorizing exact phrases, whereas Pira learned to rephrase and synthesize, resulting in a slightly higher but healthier abstractive generation loss.
Intended Use & Limitations
- Best For: Formal news articles, corporate press releases, political dispatches, and standard non-fiction prose.
- Limitations: Because the training data heavily indexes on formal journalism (Farsi-e-Ketabi), the model may struggle with highly informal social media text, slang, or deep technical/medical documents. It is also limited by the mT5 context window (max 512 tokens), meaning very long documents should be chunked before summarization.
How to Use
You can easily load Pira via the Hugging Face pipeline. For the best abstractive results, it is highly recommended to use the generation parameters shown below to prevent repetition and encourage novel sentence structures.
from transformers import pipeline
# Load the model
summarizer = pipeline("summarization", model="MahdiAlikhah/Pira")
text = """
تلسکوپ فضایی جیمز وب موفق به ثبت تصاویر جدید و خیرهکنندهای از یک کهکشان دوردست در اعماق کیهان شده است.
به گفته دانشمندان ناسا، این کهکشان که در فاصله بیش از ۱۳ میلیارد سال نوری از زمین قرار دارد، اطلاعات جدیدی درباره چگونگی شکلگیری نخستین ستارگان پس از انفجار بزرگ ارائه میدهد.
پژوهشگران امیدوارند با تحلیل دادههای طیفسنجی این تلسکوپ، به ساختار شیمیایی و گازهای موجود در این کهکشان پی ببرند.
"""
# Recommended inference parameters for Pira
summary = summarizer(
text,
max_length=128,
min_length=20,
num_beams=8, # Explores more sentence paths for better grammar
repetition_penalty=2.5, # Strongly penalizes copying exact phrases
length_penalty=1.2, # Encourages slightly longer, complete sentences
early_stopping=True
)
print(summary[0]['summary_text'])