Nyaya-Reranker-Mini-v1 — a 118M cross-encoder for Indian statute sections
cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 (Apache-2.0) fine-tuned to score a citizen's
question against a candidate section of current Indian law. It reorders the BM25
top-20 from NyayaLabs98/nyaya-statute-db
in the Nyaya retriever. One fifth the size
of the BAAI/bge-reranker-v2-m3 the project also supports, and weaker than it.
⚖️ Not legal advice. This model ranks statute sections. What to do about them is a question for an advocate enrolled under the Advocates Act, 1961.
Numbers
Full-hit recall (every gold section of a question inside the top-k) after reranking the
BM25 top-20, on the Eval-v1 questions never used to tune retrieval (n=118),
scripts/15_retrieval_recall.py --rerank:
| Ranker over BM25 top-20 | @1 | @3 | @5 | @8 | Size | CPU latency, depth 20 |
|---|---|---|---|---|---|---|
| none (BM25 order) | 45.8% | 61.0% | 74.6% | 81.4% | – | – |
| nyaya-reranker-mini-v1 | 51.7% | 70.3% | 76.3% | 82.2% | 118M | 3.2 s (Kaggle CPU) |
| bge-reranker-v2-m3 | 58.5% | 69.5% | 74.6% | 83.9% | 568M | slower |
Report: reports/retrieval_recall_rerank_mini.json. The 3.2 s CPU latency is why the
browser demo still runs BM25 only.
Training
- Examples: 21,668 (question, section text, label) triples from 4,412 training
questions: every gold section as a positive and four of the twenty BM25 hard negatives
as negatives (
scripts/41_build_retriever_pairs.py; all Eval-v1 questions excluded). - Recipe:
CrossEncoder.fit, binary relevance, batch 32, 1 epoch, 200 warm-up steps, mixed precision, one Kaggle T4 (272 s). Final training loss 0.30. - Passage text:
nyaya.rerank.passage_text(act name, section title, first 1,600 characters of the section).
Use
from sentence_transformers import CrossEncoder
model = CrossEncoder("NyayaLabs98/nyaya-reranker-mini-v1", max_length=512)
scores = model.predict([("police FIR nahi likh rahi, kya karu?", "Bharatiya Nagarik Suraksha Sanhita, 2023 — Section 173 ...")])
In the repository: python scripts/15_retrieval_recall.py --rerank NyayaLabs98/nyaya-reranker-mini-v1 --rerank-depth 20.
Limits
- Bounded by its candidates: nothing outside the BM25 top-20 can be recovered.
- Below
bge-reranker-v2-m3at k=1 by 6.8 points on n=118; use bge when you can afford it. - Trained on 27 acts plus the Constitution only.