✨ embeddinggemma-300m-taxrag-ft
The multilingual retrieval encoder for Indian tax & finance — English, Hindi, and Hinglish, one model.
Fine-tuned from google/embeddinggemma-300m
on 29.4k curated retrieval pairs from the Income-Tax Act, 2025 plus
financial-QA data. It finds the exact statutory section for a taxpayer question
— asked in English, Devanagari Hindi, or code-mixed Hinglish — with
perfect top-5 recall on Hindi and English and the highest Recall@1 and
MRR of every encoder we evaluated.
Headline results
| Metric | Score |
|---|---|
| Hindi Recall@5 | 1.000 |
| English Recall@5 | 1.000 |
| English Recall@1 | 0.990 |
| Hindi Recall@1 | 0.880 |
| Aggregate MRR@5 | 0.887 — best of all six evaluated configurations |
| Aggregate Recall@1 | 0.861 — best of all six evaluated configurations |
Measured on 180 held-out queries (100 EN / 50 HI / 30 Hinglish) against the 2,474-passage bilingual statute corpus — the same eval set that chose the checkpoint, never seen in training.
Quick start
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("vivekkopthsd/embeddinggemma-300m-taxrag-ft")
query = model.encode(
["mera HRA deduction kaise claim karu?"],
prompt_name="query", # instruction prompts preserved from base
normalize_embeddings=True,
)
doc = model.encode(
["Section 134: Deduction in respect of rent paid\n\n134. (1) ..."],
prompt_name="document",
normalize_embeddings=True,
)
sim = query @ doc.T
Trained with Matryoshka objectives — truncate embeddings to 512 or 384 dims for smaller indices and faster scoring with minimal quality loss:
query_512 = query[..., :512]
Model details
| Field | Value |
|---|---|
| Base model | google/embeddinggemma-300m (300M params, Gemma architecture) |
| Output dim | 768 (Matryoshka: 512, 384) |
| Max sequence length | 2048 |
| Prompts | query / document (registered) |
| Training | Full-parameter bf16, CachedMNRL + hard negatives, effective batch 64, LR 1e-5, 3 epochs, seed 7 |
| Languages | English · Hindi (Devanagari) · Hinglish (code-mixed) |
| License | Gemma Terms of Use (derived from a Gemma model) |
Evaluation
180 held-out queries (100 English paraphrases, 50 Hindi translations, 30 hand-written Hinglish), gold-section retrieval over the 2,474-passage bilingual corpus.
| Slice | R@1 | R@5 | MRR@5 |
|---|---|---|---|
| English | 0.990 | 1.000 | 0.995 |
| Hindi | 0.880 | 1.000 | 0.937 |
| Hinglish | 0.400 | 0.533 | 0.447 |
| Aggregate | 0.861 | 0.922 | 0.887 |
Generalization (held-out): BeIR/fiqa test R@5 0.653 vs base 0.665 — Δ −0.012. Domain fine-tuning keeps general financial-QA ability intact.
Training data
- Tier 1 — bilingual statute (19,440 pairs): every section of the Income-Tax Act, 2025 (EN + Hindi), with 3 English, 3 Hindi and 2 Hinglish query paraphrases each, paired with both the EN and the HI passage — including cross-lingual EN→HI / HI→EN alignment.
- Tier 2 — general finance (10,000 pairs): sampled from
BeIR/fiqa(CC-BY-SA-4.0). - Hard negatives: BM25 top-1 non-gold passage per query — the model learns to separate confusable sections that differ by a single word.
- Reproduce:
build_ft_pairs.py→fine_tune_encoder.py(training pairs also published as a HF dataset).
Intended use
Retrieval for RAG over Indian tax/finance documents in English, Hindi and Hinglish. Educational tool output — not legal or financial advice. Evaluated on the Income-Tax Act, 2025 corpus; expect domain shift elsewhere.
License & attribution
Derived from EmbeddingGemma-300m → Gemma Terms of Use. Training data:
MIT-licensed statute-text dataset + NLLB machine translations + BeIR/fiqa
(CC-BY-SA-4.0). The reranker used alongside this model in production
(Qwen/Qwen3-Reranker-0.6B) is Apache-2.0.