ITS Embedding BGE
An English sentence-embedding model fine-tuned for semantic search and document retrieval in the ITS Global knowledge hub.
This model maps sentences and passages to 1024-dimensional dense vectors. It is intended to be used for embedding both indexed documents and user queries in the same retrieval system.
Base model
This model is fine-tuned from BAAI/bge-large-en-v1.5.
Usage
Install Sentence Transformers:
pip install -U sentence-transformers
Load the model and generate normalized embeddings:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("X13Core/ITS_embedding_bge_1000")
texts = [
"Example document passage",
"Example user query",
]
embeddings = model.encode(
texts,
normalize_embeddings=True,
convert_to_numpy=True,
)
print(embeddings.shape) # (2, 1024)
Use the same model and preprocessing for document passages and queries. When replacing an existing embedding model, regenerate the document embeddings before switching production retrieval to this model.
Evaluation
The model was evaluated with the ITS validation set containing 1,124 queries. The reported hit rate corresponds to the evaluator's Accuracy@5 metric:
| Model | Hit rate / Accuracy@5 |
|---|---|
| Fine-tuned ITS BGE | 0.961744 |
| Stock BGE baseline | 0.916370 |
The fine-tuned model improves the measured validation result by 0.045374 absolute points on this evaluation set.
The detailed evaluation output is available in eval/Information-Retrieval_evaluation_results.csv.
Training details
- Fine-tuning objective:
MultipleNegativesRankingLoss - Similarity function: cosine similarity
- Epochs: 2
- Training batch size: 16
- Learning rate:
2e-5 - Maximum sequence length: 512 tokens
- Pooling: CLS token pooling
- Output dimension: 1024
- Embeddings are normalized for retrieval use
Intended use and limitations
This model is designed for English semantic retrieval within ITS Global. It is an embedding model, not a generative language model, and it does not produce answers by itself.
The evaluation result is specific to the available ITS validation set and retrieval configuration. It should not be interpreted as a general benchmark result or as a guarantee of performance on unrelated domains.
License
This repository is released under the MIT License. The base model and its terms should also be reviewed before redistribution or commercial use.