AriaBERT: A Pre-trained Persian BERT Model for Natural Language Understanding
Introduction
AriaBERT represents a breakthrough in natural language processing (NLP) for the Persian language. Developed to address the critical gap in efficient pretrained language models for Persian, AriaBERT is tailored to elevate the standards of Persian language tasks.
Paper: https://www.researchsquare.com/article/rs-3558473/v1
Key Features
- Diverse Training Data: AriaBERT has been trained on over 32 gigabytes of varied Persian textual data, spanning conversational, formal, and hybrid texts. This includes a rich mix of tweets, news articles, poems, medical and encyclopedia texts, user opinions, and more.
- RoBERTa Architecture: Leveraging the robustness of the RoBERTa architecture and the precision of Byte-Pair Encoding tokenizer, AriaBERT stands apart from traditional BERT-based models.
- Broad Applicability: Ideal for a range of NLP tasks including classification, sentiment analysis, and stance detection, particularly in the Persian language context.
Performance Benchmarks
- Sentiment Analysis: Achieves an average improvement of 3% over competing models.
- Classification Tasks: Demonstrates a 0.65% improvement in accuracy.
- Stance Detection: Shows a 3% enhancement in performance metrics.