Bangla-Embed-E5-Banglish
A compact 118M Bengali sentence encoder that additionally supports romanized Bengali
(Banglish) and, uniquely, retrieves across scripts — aligning romanized queries with their
native-Bengali and English counterparts. Rebased from intfloat/multilingual-e5-small by
three-stage distillation from a BGE-M3 teacher, with IndicXlit-romanized views added in the
distillation and contrastive stages.
Highlights
- Cross-script retrieval: on a held-out, human-typed romanization set (disjoint from the training transliterator) it retrieves native counterparts at acc@1 0.85 / 0.96, versus ≤0.27 for all baselines, and is the only model that retrieves across script in a controlled product-search study (MRR@10 0.31 vs ≤0.11).
- MTEB(Indic) Bengali mean 0.698 — statistically tied with the 4.8× larger BGE-M3 (0.700) and mE5-large (0.690); strongest encoder in the ≤120M tier.
For native-Bengali-only use, the companion
kazalbrur/bangla-embed-e5-small is slightly stronger
on some full-corpus retrieval sets.
Comparison — MTEB(Indic) Bengali subset
Main score per task type (mteb 2.12), single shared harness. This model is bolded.
| Model | Params | Retr | Class | Bitext-G | Bitext-C | Rerank | STS | Clust | Mean |
|---|---|---|---|---|---|---|---|---|---|
| BGE-M3 (teacher) | 568M | 0.644 | 0.879 | 0.874 | 0.722 | 0.852 | 0.593 | 0.340 | 0.700 |
| bangla-embed-e5-small-banglish (this model) | 118M | 0.791 | 0.832 | 0.826 | 0.654 | 0.840 | 0.596 | 0.349 | 0.698 |
| mE5-large | 560M | 0.631 | 0.847 | 0.876 | 0.748 | 0.852 | 0.540 | 0.339 | 0.690 |
| bangla-embed-e5-small | 118M | 0.572 | 0.848 | 0.832 | 0.668 | 0.840 | 0.554 | 0.349 | 0.666 |
| mE5-small (base) | 118M | 0.535 | 0.832 | 0.848 | 0.699 | 0.835 | 0.538 | 0.310 | 0.656 |
| LaBSE | 109M | 0.442 | 0.804 | 0.849 | 0.705 | 0.792 | 0.583 | 0.239 | 0.631 |
| Vyakyarth | 300M | 0.629 | 0.762 | 0.853 | 0.576 | 0.767 | 0.423 | 0.343 | 0.622 |
| pm-mpnet-base | 278M | 0.337 | 0.749 | 0.618 | 0.426 | 0.701 | 0.355 | 0.370 | 0.508 |
Retr=BelebeleRetrieval, Class=BengaliSentiment, Bitext-G/C=IN22 Gen/Conv, Rerank=WikipediaReranking, STS=IndicCrosslingualSTS, Clust=SIB200. This model ranks second on mean and tops cross-lingual STS; the top-3 means span only 0.010 and are statistically indistinguishable under a 7-task paired bootstrap (treat sub-0.02 gaps as ties), i.e. on par with the 4.8× larger BGE-M3.
Cross-script (Banglish) retrieval — unique to this model
acc@1 on human-typed, held-out romanized queries retrieving native counterparts (disjoint from the training transliterator):
| Model | bnₗₐₜ→en | bnₗₐₜ→bn |
|---|---|---|
| bangla-embed-e5-small-banglish (this model) | 0.85 | 0.96 |
| BGE-M3 / mE5-large / bangla-embed-e5-small | ≤0.27 | ≤0.27 |
Only this model aligns romanized Bengali with native script/English.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("kazalbrur/bangla-embed-e5-small-banglish")
# romanized (Banglish) query -> native Bengali passage
q = model.encode(["dhaka kothay?"], prompt_name="query", normalize_embeddings=True)
d = model.encode(["ঢাকা বাংলাদেশের রাজধানী।"], prompt_name="passage", normalize_embeddings=True)
print((q @ d.T)[0, 0])
E5-family conventions: prefix queries with query: and passages with passage: (handled by
prompt_name). Output dimension is 1024, L2-normalized.
Training & data
Three-stage curriculum (AdamW, cosine schedule, bf16). Distillation over ~18.7M EN–BN parallel pairs plus ~1M IndicXlit-romanized views; supervised contrastive fine-tuning (MNR) on ~2.77M pairs (Bangla-native core + SWIM-IR + MS MARCO-bn + ~0.32M romanized views); NLI polish (XNLI-bn, IndicXNLI-bn, MNLI-en). Released under the MIT license. Note that some training sources carry their own (in some cases non-commercial) terms — verify upstream data terms before commercial deployment.
Limitations
Training romanization is synthetic (IndicXlit); generalization is validated on a human-typed held-out set and a rule-based generator, but coverage is Standard Bangla only (regional typing conventions and varieties such as Sylheti/Chittagonian are unrepresented). On full-corpus MIRACL-bn/Mr.TyDi-bn it trails larger models and the untuned backbone; all metrics are automatic.