bahith-arabic-retrieval-4b
Arabic passage-retrieval embedding model (query → relevant passage), fine-tuned from Qwen/Qwen3-Embedding-4B with Matryoshka Representation Learning.
On MIRACLRetrievalHardNegatives (Arabic) it reaches 0.7971 nDCG@10, outperforming intfloat/multilingual-e5-large (0.7658) — while its embeddings stay truncatable via Matryoshka Representation Learning.
Highlights
- 🇸🇦 Arabic-first retrieval — trained on human-annotated MSA data (MIRACL, Mr. TyDi).
- 🪆 Matryoshka embeddings — truncate to any of
[2560, 1024, 512, 256, 128]dimensions (and re-normalize) with graceful quality loss. - 🎯 Instruction-aware queries — the correct query/document prompts are baked in and applied automatically.
- 📐 Output dimension: 2560 · Max sequence length: 512 · Similarity: cosine.
Results
Evaluated with MTEB on Arabic retrieval tasks (nDCG@10 / MRR@10). Baselines are run under identical conditions.
MIRACLRetrievalHardNegatives — subset ar, split dev
| Model | nDCG@10 | MRR@10 | dim |
|---|---|---|---|
| qwen3-embedding-4b-arabic-matryoshka (ours) | 0.7971 | 0.8138 | 2560 |
| qwen3-embedding-4b-arabic-matryoshka (ours) @dim=1024 | 0.7930 | 0.8091 | 1024 |
| qwen3-embedding-4b-arabic-matryoshka (ours) @dim=256 | 0.7697 | 0.7877 | 256 |
| intfloat/multilingual-e5-large | 0.7658 | 0.7989 | 1024 |
| qwen3-embedding-4b-arabic-matryoshka (ours) @dim=128 | 0.7469 | 0.7729 | 128 |
SadeemQuestionRetrieval — subset default, split test
| Model | nDCG@10 | MRR@10 | dim |
|---|---|---|---|
| intfloat/multilingual-e5-large | 0.6973 | 0.5955 | 1024 |
| qwen3-embedding-4b-arabic-matryoshka (ours) | 0.6933 | 0.5842 | 2560 |
| qwen3-embedding-4b-arabic-matryoshka (ours) @dim=1024 | 0.6911 | 0.5879 | 1024 |
| qwen3-embedding-4b-arabic-matryoshka (ours) @dim=256 | 0.6745 | 0.5652 | 256 |
| qwen3-embedding-4b-arabic-matryoshka (ours) @dim=128 | 0.6616 | 0.5612 | 128 |
Usage
Install the latest Sentence Transformers:
pip install -U sentence-transformers
Encode queries and documents (prompts are applied automatically):
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("mabdulaziz499/bahith-arabic-retrieval-4b")
# Queries and documents — the correct prompts are stored in the model and applied
# automatically by encode_query / encode_document (no manual prefixing needed).
queries = [
"ما هي عاصمة مصر؟",
"من كتب رواية موسم الهجرة إلى الشمال؟",
]
documents = [
"القاهرة هي عاصمة جمهورية مصر العربية وأكبر مدنها وأكثرها سكانًا.",
"موسم الهجرة إلى الشمال رواية للكاتب السوداني الطيب صالح صدرت عام 1966.",
"برج إيفل معلم شهير يقع في مدينة باريس عاصمة فرنسا.",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
# Cosine similarity: rows = queries, columns = documents
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
Matryoshka: smaller embeddings
# Matryoshka: truncate embeddings to a smaller dimension for cheaper storage / faster
# search. The output is re-normalized automatically, so cosine similarity still works.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("mabdulaziz499/bahith-arabic-retrieval-4b", truncate_dim=256) # any supported dim
embeddings = model.encode_document(["نص عربي للفهرسة والاسترجاع."])
print(embeddings.shape) # (1, 256)
The prompts stored in the model (and applied by encode_query / encode_document):
- Query prompt:
Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: - Document prompt:
(none)
Choosing an embedding dimension
Matryoshka lets you trade a little accuracy for much cheaper storage and faster search. Pick the smallest dimension that meets your quality bar (see the Results table above).
| dim | relative size |
|---|---|
| 2560 | 100% of full size |
| 1024 | 40% of full size |
| 512 | 20% of full size |
| 256 | 10% of full size |
| 128 | 5% of full size |
Model details
| Base model | Qwen/Qwen3-Embedding-4B |
| Output dimension | 2560 (Matryoshka: [2560, 1024, 512, 256, 128]) |
| Max sequence length | 512 |
| Similarity function | cosine |
| Language | Arabic (ar) |
| Library | sentence-transformers |
Training details
- Data: human-annotated Arabic training pairs from MIRACL and Mr. TyDi (query → positive passage).
- Hard negatives: mined with a strong teacher encoder using rank-range and relative-margin filtering to reduce false negatives.
- Loss:
MatryoshkaLosswrappingCachedMultipleNegativesRankingLoss(gradient caching for large effective batches with bounded memory), dims[2560, 1024, 512, 256, 128]. - Schedule: 3 epochs, learning rate 1e-05, cosine schedule with 10% warmup, bf16/fp16 selected by GPU.
- Model selection: best checkpoint by MIRACL-ar dev nDCG@10 (in-loop
InformationRetrievalEvaluator).
Full training, evaluation, and publishing code: github.com/m0d9/arabic-retrieval-embeddings
Intended use & limitations
- Intended for: Arabic passage / document retrieval, semantic search, and retrieval-augmented generation over Arabic corpora.
- Limitations: trained primarily on Modern Standard Arabic; dialectal coverage is limited. As with any retriever, validate on your own domain before production use.
License
Released under apache-2.0, matching the base model Qwen/Qwen3-Embedding-4B.
Acknowledgements
Built on Qwen/Qwen3-Embedding-4B; trained and evaluated with sentence-transformers and MTEB.