tax-retriever-bge-small
A retrieval embedding model fine-tuned for Indian income-tax question
answering. Built on BAAI/bge-small-en-v1.5 and adapted to the
Income-Tax Act 2025 and professional tax-guide corpus used by the
RAG Tax Assistant application.
Model
| Property | Value |
|---|---|
| Base model | BAAI/bge-small-en-v1.5 (33M params) |
| Architecture | BERT-small, mean-pooling, 512-dim embeddings |
| Max sequence length | 256 tokens |
| Training | MultipleNegativesRankingLoss, contrastive |
| Compute | Kaggle CPU kernel (no GPU), 4 epochs |
| Training pairs | 2,301 (title→content, section→content, in-doc) |
| License | Apache-2.0 (base model) |
Training data
The corpus contains 776 documents:
- 629 sections of the Income-Tax Act 2025 (machine-readable text, from the official Act consolidated text)
- 147 professional tax guides covering deductions (80C/80D/80E/80G), HRA, capital gains, ITR forms, NRI taxation, TDS and related topics
Training pairs are self-supervised: each section's title and heading are paired with its body text, so the model learns to rank the correct provision for a tax-shaped query. Word embeddings were frozen during training to preserve the base vocabulary while adapting the encoder.
Evaluation
Real-query retrieval (pilot)
20 user-style questions (not present in training pairs) were retrieved against the full corpus; a hit means the top-ranked document belongs to the question's topic family.
| Model | hits@1 | hits@3 |
|---|---|---|
| tax-retriever-bge-small | 0.85 | 0.90 |
| bge-small-en-v1.5 (base) | 0.80 | 0.85 |
| finance-retriever-bge-small | 0.75 | 0.80 |
| TF-IDF (lexical control) | 0.80 | — |
Pilot scale (n=20): the 0.85 vs 0.80 margin is one question and is not statistically significant (Clopper-Pearson CIs overlap; McNemar on n=20 cannot resolve a 1-question flip). Reported as directionally consistent, not as a measured win. A larger labeled set is the planned next step.
Section retrieval (exact match)
Leave-one-out over all 776 documents: the query is a document's title and
the target is its own content. Both models evaluated on the identical
protocol and corpus (controlled comparison; full results in
eval/tax_retriever_fixed_eval.json). This metric is close to the training
objective (title→content pairs), so it reads as a training-recall check
rather than a generalization claim.
| Model | Recall@1 | Recall@5 | MRR |
|---|---|---|---|
| tax-retriever-bge-small | 0.960 | 0.990 | 0.984 |
| bge-small-en-v1.5 (base) | 0.683 | 0.852 | 0.884 |
| Δ | +0.277 | +0.138 | +0.100 |
Baseline sanity check
A TF-IDF retriever was evaluated under the topic-family protocol as a non-neural control. It scores 0.91 hits@1 on the family metric, which is expected: the family protocol rewards lexical overlap between near-identical Act sections. The exact-match and real-query protocols above do not share this property and are the reported metrics.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("vivekkopthsd/tax-retriever-bge-small")
# BGE models require the query prefix on the query side only
queries = ["Represent this sentence for searching relevant passages: "
"How much can I deduct under Section 80C?"]
docs = ["Represent this sentence for searching relevant passages: "
"Section 80C of the Income-Tax Act, 2025."]
q_vec = model.encode(queries, normalize_embeddings=True)
d_vec = model.encode(docs, normalize_embeddings=True)
Limitations
- Trained on Indian income-tax texts; performance on other legal or financial domains is not evaluated.
- The training corpus is English; Hindi queries are encoded by an English-only tokenizer.
- Section retrieval is measured on the training corpus; cross-corpus generalization is not yet measured.
- Not legal advice. Provisions change with each Finance Act; verify against the official consolidated Act text.
Training
Reproducible training notebook: Kaggle — tax-retriever-finetune-bge-small
Training corpus dataset: India Tax Corpus