nomic-embed-bns-legal
A fine-tuned version of nomic-ai/nomic-embed-text-v1.5
specialised for retrieval over Indian criminal law — the Bharatiya Nyaya Sanhita 2023 (BNS),
Indian Penal Code 1860 (IPC), Bharatiya Nagarik Suraksha Sanhita 2023 (BNSS), and
Bharatiya Sakshya Adhiniyam 2023 (BSA).
It is built for RAG: given a natural-language legal question, it pulls the correct statutory section and distinguishes it from confusingly similar sections.
What we did
The base nomic-embed-text is a strong general-purpose embedder, but it doesn't know that a
query like "Who bears the burden of proof when a fact is within someone's personal knowledge?"
should map to BSA Section 109 and not to a different evidence section. We fine-tuned it
with contrastive triplet learning so that queries land next to the right section and away from
look-alike sections.
Dataset
11,587 contrastive triplets synthesised from the bare Acts. Each triplet is:
| Field | Content |
|---|---|
| anchor | a query (search_query: ...) — layperson scenario, professional search, or "essential ingredients" |
| positive | the correct section text (search_document: ...) |
| negative | a hard negative — a confusingly similar section from the same chapter (e.g. Theft vs Robbery vs Extortion) |
The hard, intra-chapter negatives are the key asset: they force the model to learn fine-grained legal distinctions rather than broad topic similarity. This is also why TripletLoss (which uses the curated negative directly) was chosen over in-batch-negative losses.
Training
| Hyperparameter | Value |
|---|---|
| Base model | nomic-ai/nomic-embed-text-v1.5 (137M params, 768-dim) |
| Loss | TripletLoss, cosine distance, margin 0.5 |
| Epochs | 5 |
| Batch size | 16 |
| Max sequence length | 512 (dataset max is 527 tokens; p99 ≈ 460) |
| Learning rate | 2e-5 (AdamW) |
| Warmup ratio | 0.1 |
| Precision | fp32 |
| Eval split | 5% held out (580 triplets) |
| Seed | 42 |
| Trainer | SentenceTransformerTrainer |
| Hardware | NVIDIA RTX 5090 |
Results
Triplet accuracy on the held-out split (does the model rank the correct section above the hard negative?):
| Model | Triplet accuracy |
|---|---|
Base nomic-embed-text-v1.5 | 0.9552 |
| This model | 0.9983 |
Usage
Important: keep nomic's task prefixes —
search_query:for questions,search_document:for the sections you index. Retrieval quality drops noticeably without them.
sentence-transformers
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("saitejasathiraju/nomic-embed-bns-legal", trust_remote_code=True)
q = m.encode("search_query: What constitutes murder under the new criminal code?")
d = m.encode("search_document: Section 101 of the Bharatiya Nyaya Sanhita 2023 — Murder ...")
Ollama
ollama pull shb/legal-embed
curl http://localhost:11434/api/embed -d '{
"model": "shb/legal-embed",
"input": "search_query: What constitutes murder under the new criminal code?"
}'
Limitations
- Triplet accuracy uses one curated negative per query; real RAG ranks against the full section corpus, so treat 0.9983 as a sanity signal, not end-to-end RAG recall.
- English queries only. Covers BNS / IPC / BNSS / BSA; not trained on case law or other statutes.