bge-m3-uzbek-cyrillic
A fine-tuned version of BAAI/bge-m3 optimized for semantic similarity and retrieval over Uzbek (Cyrillic) text, with a focus on legal and formal domains. The model produces dense embeddings suitable for search, clustering, and retrieval-augmented generation while retaining the multilingual coverage of the base model.
Model Details
| Property | Value |
|---|---|
| Base model | BAAI/bge-m3 |
| Parameters | 0.6B |
| Embedding dimension | 1024 |
| Max sequence length | 8192 tokens |
| Fine-tuning objective | Contrastive learning |
| Precision | float32 |
| Languages | Uzbek (Cyrillic), Russian, English |
| License | Apache 2.0 |
Intended Use
- Semantic search and dense retrieval
- Retrieval-augmented generation (RAG) over Uzbek documents
- Document and sentence similarity detection
- Paraphrase evaluation for closely related sentence pairs
- Duplicate detection and clustering
Limitations
- The fine-tune targets Cyrillic-script Uzbek; performance on Latin-script Uzbek may differ.
- The model is optimized for legal and formal text and may be less accurate on informal or conversational language.
- Embedding quality depends on input preprocessing and normalization being consistent between indexing and querying.
Usage
sentence-transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Just-Bax/bge-m3-uzbek-cyrillic")
sentences = [
"Ўзбекистон Республикаси Конституцияси олий юридик кучга эга.",
"Конституция давлатнинг асосий қонуни ҳисобланади.",
"Бугун ҳаво очиқ ва қуёшли.",
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities)
transformers (manual pooling)
from transformers import AutoTokenizer, AutoModel
import torch
import torch.nn.functional as F
model_id = "Just-Bax/bge-m3-uzbek-cyrillic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id)
text = "Ўзбекистон Республикаси қонунларига мувофиқ ..."
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
with torch.no_grad():
outputs = model(**inputs)
# CLS pooling followed by L2 normalization
embedding = outputs.last_hidden_state[:, 0]
embedding = F.normalize(embedding, p=2, dim=1)
print(embedding.shape)
Training
The model was fine-tuned from BAAI/bge-m3 with a contrastive learning objective on Uzbek (Cyrillic) text drawn from legal and formal sources. CLS pooling and L2 normalization are used to produce the final sentence embeddings.
License
Released under the Apache 2.0 license, consistent with the base model BAAI/bge-m3.
Citation
If you use this model, please cite the base model:
@misc{bge-m3,
title = {BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation},
author = {Chen, Jianlv and Xiao, Shitao and Zhang, Peitian and Luo, Kun and Lian, Defu and Liu, Zheng},
year = {2024},
url = {https://huggingface.co/BAAI/bge-m3}
}