bge-m3-ir-research-lora-v1
LoRA fine-tune of BAAI/bge-m3 for dense retrieval over information-retrieval research literature (BM25, DPR, ColBERT, RAG, MS MARCO, BEIR). Weights below are the merged model (base + adapter) — no peft needed to load it.
Usage
from FlagEmbedding import BGEM3FlagModel
model = BGEM3FlagModel("Lzvick/bge-m3-ir-research-lora-v1", use_fp16=True)
output = model.encode(
["What is dense passage retrieval?"],
return_dense=True,
return_sparse=True,
return_colbert_vecs=False,
)
Training
| Setting | Value |
|---|---|
| Base model | BAAI/bge-m3 |
| Hardware | Single consumer GPU, 12GB VRAM |
| Method | LoRA (r=16, alpha=32, dropout=0.1, targets: query,key,value) |
| Trainable params | 2,359,296 / 570,114,048 (0.41%) |
| Epochs | 3 |
| Effective batch size | 128 (4 per-device × 32 grad-accum) |
| Learning rate | 1e-5 |
| Query / passage max len | 256 / 256 |
| Train group size | 8 (1 positive + 7 negatives per query) |
| Precision | bf16 |
| Train set | 2,072 examples |
Evaluation
Sanity check on 109 held-out validation examples: mean cosine-similarity margin between query→positive and query→negative pairs.
| Model | Mean pos-neg margin |
|---|---|
Baseline (BAAI/bge-m3) | 0.0849 |
| Fine-tuned (this model) | 0.0861 |
This is a coarse sanity check, not a retrieval benchmark (no nDCG/Recall/MRR test-set evaluation has been run yet). Treat this as an early checkpoint from a short run (~51 optimizer steps), not a fully validated release.