BGE-M3 Bangla Cross-Script Fine-tuned
A fine-tuned version of BAAI/bge-m3 optimized for Bangla cross-script semantic search. The model understands and aligns text across:
- Pure Bangla (বাংলা)
- Banglish (Romanized Bangla)
- Mixed code-switching (Bangla + English)
- Informal/colloquial Bangla
- Noisy/typo-heavy text
- English (same intent)
Key Results
| Metric | Base BGE-M3 | Fine-tuned | Improvement |
|---|---|---|---|
| Avg cross-script similarity | 0.689 | 0.788 | +0.098 |
| Bangla ↔ Banglish | 0.652 | 0.861 | +0.209 |
| Banglish ↔ English | 0.680 | 0.919 | +0.239 |
| Typos ↔ English | 0.612 | 0.856 | +0.244 |
| Dev triplet accuracy | 95.30% | 98.74% | +3.44% |
| Retrieval Hit@1 | 1.000 | 1.000 | — |
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("nahidstaq/bge-m3-bangla")
# These all mean the same thing - the model knows that!
sentences = [
"আজকের আবহাওয়া কেমন?", # Pure Bangla
"ajker weather kemon?", # Banglish
"আজকে weather কেমন হবে?", # Mixed
"what is today's weather?", # English
]
embeddings = model.encode(sentences, normalize_embeddings=True)
# Compute similarity
from sentence_transformers.util import cos_sim
print(cos_sim(embeddings, embeddings))
# All pairs will show high similarity (>0.85)
Training Details
- Base model: BAAI/bge-m3 (568M params, XLM-RoBERTa)
- Method: LoRA fine-tuning (rank=32, alpha=64)
- LoRA targets: query, key, value, dense layers
- Loss: MultipleNegativesRankingLoss (temperature=0.05)
- Training data: 39,082 examples
- 27,082 augmented cross-script pairs (6 script variants per intent)
- 12,000 synthetic cross-script parallel pairs generated using Gemini 2.5 Flash and Claude Opus 4.5
- 10 domains: food, travel, health, tech, education, sports, society, ecommerce, government, agriculture
- Epochs: 3
- Batch size: 8
- GPU: NVIDIA RTX 4090 (24GB)
- Training time: ~3 hours
- Best checkpoint: 98.74% triplet accuracy on dev set
Intended Use
- Semantic search over Bangla content (government services, FAQs, documents)
- Cross-script information retrieval (user types in Banglish, retrieves Bangla docs)
- Bangla chatbot retrieval-augmented generation (RAG)
- Bangla text clustering and deduplication across scripts
Limitations
- Trained primarily on Bangladesh-specific domains (government, daily life, services)
- Within-group coherence is lower than base model (by design — embeddings are more discriminative)
- Pure Bangla ↔ informal Bangla similarity slightly decreased (-0.12)
- Not optimized for long documents (best for queries and short passages)
Credits
Built by Md Nahid Hasan (@nahidstaq | nahidscript.com).
Based on BAAI/bge-m3.
Citation
@misc{bge-m3-bangla-cross-script,
author = {Hasan, Md Nahid},
title = {BGE-M3 Bangla Cross-Script Fine-tuned},
year = {2025},
url = {https://huggingface.co/nahidstaq/bge-m3-bangla},
note = {Fine-tuned on cross-script Bangla data for semantic search}
}