tax-reranker-minilm
A cross-encoder reranker for questions about India's Income-Tax Act, 2025, fine-tuned from
cross-encoder/ms-marco-MiniLM-L-6-v2
(22M parameters). It reorders passages retrieved by a first-stage retriever and is used as the
second stage of TaxRAG.
Training
- Data: questions written from the content of each section of the Act by Qwen3-4B-Instruct (not from section headings). Questions that mention a section number or mostly repeat the heading were removed. Sections were split into disjoint train/test groups; this model saw only questions from the 327 training sections.
- Pairs: 2,765 (each training question with its gold passage as the positive and 4 hard negatives: the top passages from other sections returned by the FiQA-tuned bge-small retriever).
- Objective: binary cross-entropy on (question, passage) pairs, 2 epochs, AdamW lr 2e-5, batch 16, max length 384, fp16, one T4 GPU.
Evaluation
277 questions from 150 held-out sections (never seen in training); gold = the section a question was written from; section-level metrics over 1,237 passages.
| Pipeline | Recall@1 | Recall@5 | MRR@10 |
|---|---|---|---|
| Dense only (FiQA bge-small) | 0.502 | 0.773 | 0.632 |
| Dense top-30 → ms-marco-MiniLM (base) | 0.726 | 0.903 | 0.796 |
| Dense top-30 → this model | 0.758 | 0.906 | 0.820 |
| Dense + BM25 (RRF) top-30 → this model | 0.773 | 0.910 | 0.833 |
Both rerankers improve strongly on dense retrieval (exact McNemar on Recall@5 vs dense only, p < 1e-8). Over the base ms-marco reranker the gain is modest (+0.024 MRR@10).
Limitation. On an older, easier test set of 100 questions paraphrased from section headings, this model is weaker than the base reranker (MRR@10 0.843 vs 0.907): it was tuned on content-style questions and generalises less well to heading-style phrasing.
Usage
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("vivekkopthsd/tax-reranker-minilm")
model = AutoModelForSequenceClassification.from_pretrained("vivekkopthsd/tax-reranker-minilm").eval()
query = "If I receive rent this year that was due in earlier years, how is it taxed?"
passages = ["Arrears of rent and unrealised rent ...", "Determination of annual value ..."]
enc = tok([query] * len(passages), passages, truncation=True, max_length=256, padding=True, return_tensors="pt")
with torch.no_grad():
scores = model(**enc).logits.squeeze(-1)
ranked = [passages[i] for i in scores.argsort(descending=True)]
On CPU, reranking 15 candidates at 256 tokens takes about 2.5 s on a 4-core laptop.
Intended use
Ranking passages of the Income-Tax Act, 2025 for question answering. Not legal or tax advice.