sanskritgemma-256
A Model2Vec static distillation of
Ganaraj/rgveda-embedding-gemma — an
embeddinggemma variant tuned on Vedic Sanskrit. Static embeddings are a lookup table: no
transformer runs at inference, so encoding is ~1,000× faster on CPU at the cost of word order and
context.
- 256 dimensions, float16, PCA-256, mean pooling, normalized — ~131 MB
- Tokenizer:
Ganaraj/rgveda-embedding-gemma - Sibling widths at 384, 512 and 768 exist; see the Matryoshka note below before reaching for them
Usage
from model2vec import StaticModel
model = StaticModel.from_pretrained("karthikrajgopal/sanskritgemma-256")
embeddings = model.encode(["धर्मक्षेत्रे कुरुक्षेत्रे", "the field of dharma"])
Evaluation
Measured on the complete Bhagavad Gītā — 701 verses indexed as Devanagari only, queried by their IAST transliteration and by two independent published English translations. An English query is therefore genuine cross-lingual retrieval with no leakage. Scores are MRR@10.
Against other static models, encoder only
| model | teacher | dims | IAST | EN close | EN free | mean |
|---|---|---|---|---|---|---|
| potion-multilingual-128M | — | 256 | 0.647 | 0.265 | 0.233 | 0.536 |
| distilled-embeddinggemma | embeddinggemma-300m | 256 | 0.684 | 0.251 | 0.205 | 0.535 |
| (same teacher, 512d) | embeddinggemma-300m | 512 | 0.677 | 0.243 | 0.203 | 0.531 |
| sanskritgemma-256 | rgveda-embedding-gemma | 256 | 0.656 | 0.317 | 0.249 | 0.555 |
The teacher is the lever, not the width. Doubling the general embeddinggemma distillation to 512 dims made it worse; re-distilling from a Sanskrit-tuned teacher at 256 dims moved English-close retrieval from 0.251 to 0.317.
Matryoshka: the widths are nearly flat
| width | size | mean MRR |
|---|---|---|
| 256 | 131 MB | 0.555 |
| 384 | 196 MB | 0.561 |
| 512 | 262 MB | 0.563 |
| 768 | 393 MB | 0.565 |
256 reaches 98% of 768's quality at a third of the disk, which is why this is the width worth publishing.
When to use it, and when not to
This model is strongest where the encoder is doing the retrieval: flat chunk search, an ANN-only path, or a transliteration-heavy corpus — its IAST top-1 (0.487) is the best of the static models tested, against potion's 0.420.
Measured inside a full retrieval stack — tree structure, section roll-up and assembled context —
potion-multilingual pulled ahead (0.238 against 0.213 mean MRR, paired bootstrap
95% CI [+0.016, +0.074] over 500 queries). Once whole sections are being assembled, what is left
for the encoder is loose English semantics, and a broadly-trained multilingual model handles that
better. Benchmark the configuration you ship, not the encoder alone.
Static pooling also cannot recover a word that sandhi or compounding has fused: dharmakṣetre is
one token. In litesearch that gap is closed outside the
encoder, by indexing lemmas and Monier-Williams glosses alongside the text.
Provenance
Distilled with Model2Vec. Full evaluation, including the
paired significance test, is in litesearch's 10_sanskrit_eval notebook.
MIT, following the Model2Vec tooling. The teacher derives from Google's embeddinggemma, whose Gemma terms of use flow through — check those before redistributing derivatives.