EmbeddingGemma 300M Norwegian Health (CachedMNRL Fine-tuned)
A Norwegian health-domain embedding model fine-tuned with CachedMultipleNegativesRankingLoss combining explicit hard negatives with in-batch negatives for optimal ranking performance.
Quick Start
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('thivy/embeddinggemma-300m-norwegian-health-cachedmnrl-v1')
# Encode texts
query = "Hva er symptomene på diabetes?"
documents = [
"Diabetes gir tørste, hyppig vannlating og tretthet.",
"Høyt blodtrykk kan gi hodepine."
]
embeddings = model.encode([query] + documents)
similarity = model.similarity(embeddings[0], embeddings[1:])
print(similarity)
Model Details
- Base Model: google/embeddinggemma-300m
- Embedding Size: 768
- Max Tokens: 2048
- Language: Norwegian
- Domain: Health & Medical
Training
Dataset: 330,120 triplets (anchor, positive, negative) from Norwegian health documents
Configuration:
- Loss: CachedMultipleNegativesRankingLoss (InfoNCE + GradCache)
- Learning Rate: 1.5e-5
- Logical Batch Size: 192 (96 per GPU × 2 GPUs)
- Mini-batch Size: 24 (GradCache)
- Epochs: 1
- Hardware: 2× A100 80GB with DDP
Key Innovation: CachedMNRL uses GradCache to enable large logical batch sizes (more in-batch negatives) while keeping GPU memory manageable. Each anchor sees its explicit hard negative plus all in-batch samples as candidates, combining the best of both approaches.
Training Lineage
| Model | Loss Function | NDCG@10 |
|---|---|---|
| Baseline (google/embeddinggemma-300m) | — | 56% |
| Pairs Fine-tuned (v1) | MultipleNegativesRankingLoss | 69% |
| Triplet Fine-tuned (v1) | TripletLoss | 71% |
| CachedMNRL (this model) | CachedMultipleNegativesRankingLoss | TBD |
Use Cases
- Medical question answering
- Health information retrieval
- RAG systems for healthcare
- Semantic search in Norwegian medical texts
Limitations
- Optimized for Norwegian health/medical content
- May not generalize well to other domains or languages
- Best with sufficient context (full sentences/paragraphs)
Citation
@misc{embeddinggemma-300m-norwegian-health-cachedmnrl,
title={EmbeddingGemma 300M Norwegian Health CachedMNRL Fine-tuned},
author={Thivy},
year={2026},
url={https://huggingface.co/thivy/embeddinggemma-300m-norwegian-health-cachedmnrl-v1}
}
License
Inherits license from base EmbeddingGemma model (Gemma license).