ecommerce-product-search-embeddings-base
The quality variant of the ecommerce-product-search-embeddings family: 109M parameters, 768-dim Matryoshka embeddings, fine-tuned from BAAI/bge-base-en-v1.5 on the same 427,655 real Amazon ESCI relevance judgments with the same audited recipe. Pick this over the 33M flagship when recall over a large catalog matters more than encode throughput.
When should I pick this over the small model?
Honest paired comparison on the held-out ESCI test split (8,956 queries, identical protocol):
| small (33M) | this model (109M) | verdict | |
|---|---|---|---|
| nDCG@10, judged pools | 0.7483 | 0.7521 | paired delta +0.0037, CI [+0.0013, +0.0061] — real but small |
| Recall@100, 1.22M-product corpus | 0.5212 | 0.5420 | +2.1 points, non-overlapping CIs — the real difference |
| MRR@10, full corpus | 0.4220 | 0.4385 | clear win |
| CPU throughput (queries/s) | 2,135 | 645 | 3.3× slower — the price |
The pattern is exactly what model capacity buys in retrieval: ranking a shortlist barely improves, but finding the right products in a 1.2M-item haystack improves meaningfully. If your catalog is a few thousand products, keep the small model. If it's six figures and recall is your bottleneck — or you batch-encode offline where 645 q/s is still plenty — this is the better tool.
How do I use it (copy-paste)?
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("albertobarnabo/ecommerce-product-search-embeddings-base")
q = model.encode_query(["pan that doesnt stick eggs"]) # prompt baked in
d = model.encode_document(["T-fal Nonstick Fry Pan 12 Inch"])
print(model.similarity(q, d))
Matryoshka: shrink the index without changing models
Trained with MatryoshkaLoss over [768, 384, 128, 64] — truncate + renormalize:
| dims | nDCG@10 | vs the 33M model |
|---|---|---|
| 768 (native) | 0.7521 | — |
| 384 | 0.7504 | ≈ small@384 (0.7483) at the same index size |
| 128 | 0.7446 | > small@128 (0.7423) |
| 64 | 0.7372 | > small@64 (0.7327), and above zero-shot bge-small@384 (0.7250) at 1/12th the width |
model = SentenceTransformer("albertobarnabo/ecommerce-product-search-embeddings-base", truncate_dim=128)
Training & evaluation
Identical audited pipeline to the flagship — same 115,631 real pairs + labeled synthetic noisy-query augmentation, rotating graded hard negatives (2 Substitute + 1 Irrelevant), GradCache batch 256, cosine schedule to convergence (dev peak at step 1050 of 1800), checkpoint selection on the published graded metric, zero train/test query overlap. Differences: lr 2e-5, Matryoshka dims [768, 384, 128, 64]. Full protocol, leak guards, and limitations: see the flagship card — everything there applies.
Typo-robustness note (same synthetic-noise dev eval as the family): clean 0.7540 → noised 0.7196, a −3.4-point degradation — same pattern as every model in this family, still worse than MiniLM's −1.9; we say so rather than hide it.
Citation
@article{reddy2022shopping,
title={Shopping Queries Dataset: A Large-Scale {ESCI} Benchmark for Improving Product Search},
author={Reddy, Chandan K. and M{\`a}rquez, Llu{\'i}s and Valero, Fran and Rao, Nikhil and Zaragoza, Hugo and Bandyopadhyay, Sambaran and Biswas, Arnab and Xing, Anlu and Subbian, Karthik},
journal={arXiv preprint arXiv:2206.06588},
year={2022}
}
Family: flagship 33M · reranker · static ~31MB · dataset