For more details please refer to our research paper and GitHub repo
BGE-Finetuned (paper, code)
In this project, we fine-tuned the BGE-M3 dense embeddings model on three low-resource Nigerian languages (Yoruba, Hausa, and Igbo). We also enabled cross-lingual retrieval by finetuning the model on English queries mapped to each of the three languages.
Specs
- Model
| Model Name | Dimension | Sequence Length | Introduction |
|---|---|---|---|
| abdulmatinomotoso/bge-m3-finetuned | 1024 | 8192 | multilingual; contrastive learning from bge-m3 |
- Data
| Dataset | Description |
|---|---|
| bge-m3-finetuned-data | Fine-tuning data used by bge-m3-finetuned |
Usage
import torch
from FlagEmbedding import BGEM3FlagModel
device = "cpu"
if torch.cuda.is_available():
device = "cuda"
elif torch.backends.mps.is_available():
device = "mps"
half_precision = False # or True if you really want
# Load the abdulmatinomotoso/bge-finetuned model
model = BGEM3FlagModel('abdulmatinomotoso/bge-finetuned', use_fp16=half_precision, devices=[device])
# Example text to generate embeddings for
documents = [
"Eyi jẹ́ gbolohun àpẹẹrẹ",
"Eyi kì í ṣe gbolohun àpẹẹrẹ",
"Nke a bụ nkebisiokwu atụ",
"Nke a abụghị nkebisiokwu atụ",
"Wannan jimla ce ta misali",
"Wannan ba jimla ce ta misali ba"
]
query = "Where is the example?"
# Generate embeddings
sparse_embeddings = False # The sparse embeddings are not useful as is, they will require some work to make the align with the entire model.
multivec_embeddings = False # The multivector embeddings are still useful despite training only for dense, but this sample uses only dense embeddings
dense_embeddings = True
doc_embeddings = model.encode(documents, return_sparse=sparse_embeddings, return_dense=dense_embeddings, return_colbert_vecs=multivec_embeddings)["dense_vecs"]
query_embeddings = model.encode([query], return_sparse=sparse_embeddings, return_dense=dense_embeddings, return_colbert_vecs=multivec_embeddings)["dense_vecs"]
similarity_scores = query_embeddings @ doc_embeddings.T
print(similarity_scores)
# array([[
# 0.4360781, # Higher similarity than the opposite text
# 0.40966046,
# 0.4775736, # Higher similarity than the opposite text
# 0.44197953,
# 0.4514127, # Higher similarity than the opposite text
# 0.41557986
# ]], dtype=float32)
Acknowledgement
Thanks to the authors of open-sourced datasets, including Miracl, and Wura. Thanks to the open-sourced libraries like Trafilatura, FlagEmbedding.
Citation
If you find this repository useful, please consider giving a star :star: and citation
@inproceedings{omotoso-etal-2025-improving,
title = {Improving BGE-M3 Multilingual Dense Embeddings for Nigerian Low Resource Languages},
author = {Omotoso, Abdulmatin and Shopeju, Habeeb and Joshua, Adejumobi Monjolaoluwa and Oni, Shiloh},
booktitle = {Proceedings of the 9th Widening NLP Workshop},
year = {2025},
month = nov,
address = {Suzhou, China},
publisher = {Association for Computational Linguistics},
pages = {224--229},
doi = {10.18653/v1/2025.winlp-main.33},
url = {https://aclanthology.org/2025.winlp-main.33/}
}