Financial Embedding — BGE-small fine-tuned on FinancialPhraseBank
Fine-tuned sentence-embedding model for financial text retrieval. Upgrades semantic search / RAG retrieval over financial sentences (news, invoices, contracts) by learning sentiment-aware similarity.
Model
- Base:
BAAI/bge-small-en-v1.5(~33M params) - Method: contrastive fine-tuning with MultipleNegativesRankingLoss
- Training pairs: 4,356 same-label sentence pairs from FinancialPhraseBank (Malo et al. 2014; mirrored on Kaggle)
- Hyperparameters: 3 epochs, batch 32, lr 2e-05, max_seq 128
- Hardware: Tesla T4 (Kaggle GPU)
- Word-embedding layer frozen during training for stability
Evaluation (leave-one-out label retrieval, held-out 483 sentences)
| Metric | Base | Fine-tuned | Δ |
|---|---|---|---|
| Recall@1 | 0.6729 | 0.8282 | +0.1553 |
| MRR | 0.7995 | 0.8848 | +0.0853 |
Pooled-200 subsample: Recall@1 0.6150 → 0.8100, MRR 0.7684 → 0.8784 (base → fine-tuned).
A retrieval is a hit when the top-1 neighbor of a sentence shares its financial-sentiment label (positive/neutral/negative).
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("vivekkopthsd/financial-embedding-bge-small")
emb = model.encode(["Earnings per share amounted to a loss of EUR 0.38."])
Limitations
- Evaluated on a label-retrieval proxy, not human-judged search relevance.
- English only; vocabulary limited to financial-news language.
- Full eval script: Kaggle notebook
financial-embedding-finetune-bge-small(vivekkopthsd).