SA-Retrieval-Embeddings-0.2B
Saudi Arabic Retrieval-Optimized Sentence Embeddings
This model is a retrieval-optimized SentenceTransformer, fine-tuned from Omartificial-Intelligence-Space/SA-STS-Embeddings-0.2B, and specifically designed for:
- Semantic retrieval
- RAG (Retrieval-Augmented Generation)
- Paragraph-level semantic search
- Chunk-based document retrieval
- Saudi Arabic dialect understanding
Unlike general semantic similarity models, this model is explicitly trained to rank the correct semantic chunk at the top, even among closely related alternatives.
🔍 What makes this model different?
Most Arabic embedding models are trained on pairwise similarity only.
This model goes further by incorporating:
- Summary → Chunk retrieval supervision
- Hard negatives from semantic chunk boundaries
- Triplet-based discrimination
- In-batch negatives via MNLR
As a result, it excels in real-world retrieval scenarios, not just sentence similarity.
🧠 Training Overview
- Base Model: SA-STS-Embeddings-0.2B
- Training Objective:
- MultipleNegativesRankingLoss (primary)
- TripletLoss with hard negatives (boundary-based)
- Embedding Dimension: 768
- Pooling Strategy: Mean pooling
- Max Sequence Length: 512 tokens
- Training Samples: 4,038+ supervised retrieval examples
- Precision: FP16
Training Data
The model was trained using Saudi Semantic Chunking data, where:
- Each document is split into 3–5 semantic chunks
- Each chunk has a human-written summary
- Retrieval task:
summary → correct chunk among other chunks from the same document
Dataset: 👉 Omartificial-Intelligence-Space/Saudi-Semantic-Chunks
📊 Evaluation Results
The model was evaluated on a hard retrieval benchmark consisting of
1,515 retrieval cases across 24 Saudi domains, using chunk-level negatives.
🏆 Leaderboard Comparison
Key Takeaways
- Best Top-1 Accuracy → correct chunk ranked first ~88% of the time
- Best MRR → correct chunk appears very early in ranking
- Excellent Recall@5 (99.2%) → ideal for RAG pipelines
- Highest FinalScore → best overall balance of retrieval + discourse awareness
📐 Metric Definitions
- Top-1: Correct chunk ranked first
- MRR: Mean Reciprocal Rank
- Recall@k: Correct chunk appears in top-k
- nDCG: Ranking quality with position discount
- Contrast: (Intra-chunk similarity − Inter-chunk similarity)
- FinalScore: 0.4 × Top-1 + 0.3 × MRR + 0.2 × Contrast + 0.1 × nDCG
🧪 Usage
Install
pip install -U sentence-transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"Omartificial-Intelligence-Space/SA-Retrieval-Embeddings-0.2B"
)
sentences = [
"أفضل وقت لزيارة العلا في الشتاء",
"العلا تكون أجمل في الشتاء والجو معتدل",
"زحمة الرياض اليوم غير طبيعية"
]
embeddings = model.encode(sentences, normalize_embeddings=True)
from sklearn.metrics.pairwise import cosine_similarity
query = "أفضل وقت لزيارة أبها"
chunks = [
"أبها تتميز بأجواء معتدلة في الصيف.",
"الرياض مدينة مزدحمة.",
"مطاعم جدة متنوعة."
]
q_emb = model.encode(query, normalize_embeddings=True)
c_embs = model.encode(chunks, normalize_embeddings=True)
scores = cosine_similarity([q_emb], c_embs)[0]
for s, c in sorted(zip(scores, chunks), reverse=True):
print(round(s, 3), c)
🎯 Intended Use
- RAG systems
- Semantic search engines
- Knowledge base retrieval
- Document chunk retrieval
- Saudi dialect applications
- Government & enterprise search
⚠️ Limitations
- Optimized for Saudi Arabic (dialect + MSA)
- Not trained for cross-lingual retrieval
- Not intended for generative tasks
- Best performance when text is chunked semantically
@misc{sa_retrieval_embeddings_2025,
title = {SA-Retrieval-Embeddings-0.2B: Retrieval-Optimized Saudi Arabic Sentence Embeddings},
author = {Omer Nacar},
year = {2025},
publisher = {HuggingFace}
}