Qwen3-Embedding-0.6B — Arabic E-commerce Search (fine-tuned)
A fine-tune of Qwen/Qwen3-Embedding-0.6B
for dense retrieval of Arabic e-commerce products. Given an Arabic shopping
query, the matching product embeds closest in vector space.
Trained with contrastive learning (InfoNCE) + LoRA on
prestoai/arabic-ecom-data.
The LoRA adapter has been merged into the base — this is a standalone model,
no peft needed at load time.
⚠️ Early checkpoint. This is the step-500 of 1500 checkpoint (~33% of the planned run, ~2.3% of one epoch / ~16k pairs seen). It already beats the base model on every metric; a fuller run is expected to improve it further.
Results — before vs after fine-tuning
Dense retrieval on a held-out subset of ArabicEcomSearchData (2,901 queries =
10% sample; corpus = 22,559 judged items + 20,000 random distractors = 42,559
products), scored with the dataset's official evaluate.py. Base and
fine-tuned models were evaluated on the identical queries and corpus.
| Metric | Base | Fine-tuned | Δ | Relative |
|---|---|---|---|---|
| nDCG@10 | 0.4886 | 0.5442 | +0.0556 | +11.4% |
| Recall@10 | 0.4515 | 0.4970 | +0.0455 | +10.1% |
| MRR@10 | 0.5818 | 0.6395 | +0.0577 | +9.9% |
| Success@10 (HitRate) | 0.7790 | 0.8352 | +0.0562 | +7.2% |
| Precision@10 | 0.2474 | 0.2721 | +0.0247 | +10.0% |
| nDCG@50 | 0.5582 | 0.6122 | +0.0540 | +9.7% |
| Recall@50 | 0.6739 | 0.7225 | +0.0486 | +7.2% |
Read the numbers honestly: the subset uses fewer distractors than the full 107k-product corpus, so the absolute values run slightly optimistic versus the dataset's published full-corpus Meilisearch baseline (nDCG@10 = 0.6241). The trustworthy signal is the base-vs-fine-tuned delta — both models saw exactly the same data, and the fine-tune improves every metric by ~7–11%.
During-training validation (in-batch IR evaluator, 1,000 queries) at step 500:
accuracy@1 = 0.647, accuracy@10 = 0.946, nDCG@10 = 0.800, MRR@10 = 0.752.
Usage
This is an asymmetric retrieval model: wrap queries in the instruction
prompt; embed documents (products) as-is. The query prompt is stored on the
model as prompt_name="query".
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("prestoai/qwen3-embedding-0.6b-arabic-ecom")
queries = ["حذاء رياضي رجالي", "سماعات بلوتوث لاسلكية"]
products = ["حذاء رياضي للرجال نايك", "ساعة يد ذكية", "سماعة أذن بلوتوث لاسلكية"]
q = model.encode(queries, prompt_name="query", normalize_embeddings=True) # queries -> instruction
d = model.encode(products, normalize_embeddings=True) # documents -> plain
scores = q @ d.T # cosine similarity; argmax per row = best product
The full query instruction baked into the model is:
Instruct: Given an Arabic e-commerce search query, retrieve the product that best matches it
Query: {query}
Documents are rendered consistently as name | brand | categories | attributes | description
(empty / placeholder fields dropped).
Training
- Base:
Qwen/Qwen3-Embedding-0.6B(last-token pooling + L2 normalize) - Objective:
CachedMultipleNegativesRankingLoss(InfoNCE with in-batch + mined hard negatives; GradCache enables large effective batches) - Data (two subsets, trained jointly):
positives→(query, matching product)— 578,690 train rows (in-batch negatives)pairs_with_negatives→(query, product, hard negative)— 130,903 train rows
- PEFT: LoRA
r=16,alpha=32,dropout=0.05,target_modules=all-linear(q/k/v/o/gate/up/down proj),task_type=FEATURE_EXTRACTION— ~10M trainable params (~1.6% of the base) - Optim: lr
1e-4, cosine schedule, 5% warmup, batch 32, mini-batch 8, max_seq_length 128, seed 42 - Steps: 500 / 1500 (stopped early); fp32 on Apple Silicon (MPS)
- Split: train/val/test by hash of query text (98/1/1) — no query leaks across splits
Limitations
- Early checkpoint (33% of planned training).
- Tuned for Arabic product search; other languages/domains untested here.
- Reported metrics are on a benchmark subset (see the results note above).
Citation / links
- Base model: Qwen/Qwen3-Embedding-0.6B
- Dataset: prestoai/arabic-ecom-data
- Framework: sentence-transformers 5.6 · PEFT 0.19