prabhasa-nyaya element judge 4B v0
Fine-tuned Qwen/Qwen3-4B for element-level legal judgment on Indian statute scenarios. Given a statute text, a scenario, and a list of extracted facts, the model decides whether each statutory element is established (supported by a specific fact) or not_established.
Intended use
Research prototype for structured legal element analysis under 14 Indian Penal Code and related statutes. Not a legal advice tool.
Output format
established <fact_id>:<start>:<end> # element is established, citing the supporting fact span
not_established # element is not supported by the given facts
Covered statutes (14 contracts)
IPC §§ 302, 304A, 376, 378, 379, 380, 384, 390, 392, 405, 415, 420; NDPS Act § 22; Prevention of Corruption Act § 7.
Training data
1,463 element-judgment rows from 14-contract scenarios (T-tier teacher examples + confusable-negative pairs). Leak-certified: 0 containment hits at IDF-weighted 0.9 threshold against the 1,519-item held-out eval set. Minority class (not_established) oversampled 1:1 per epoch. No eval items appear in training.
Evaluation (signed, pipeline-measured)
Evaluated on 1,519 held-out element-judgment items. No few-shots. Greedy decode (threshold = 0.5).
| Partition | n | Accuracy | established recall | not_established recall | not_established precision |
|---|---|---|---|---|---|
| negatives | 563 | 0.876 | 0.867 | 0.886 | 0.846 |
| full_ir_gold | 956 | 0.948 | 0.948 | n/a¹ | n/a¹ |
| overall | 1519 | 0.921 | — | — | — |
¹ The full_ir_gold partition contains only established elements; not_established recall is undefined there.
Prompted baselines on the same 1,519-item eval (6-shot, all base models unless noted):
| Model | not_established recall (negatives) |
|---|---|
| Qwen3-4B-Base | 0.125 |
| Qwen3-4B (post-trained / Instruct) | 0.224 |
| Qwen3-14B-Base | 0.392 |
| Qwen2.5-32B-Base | 0.427 |
| This model — trained, no shots | 0.886 |
Dual-signed by Track A and Track B reviewers.
Limits
- Covers 14 statutes only. Performance on other Indian law statutes is untested.
- Research prototype — evaluation was conducted on a fixed held-out set from the same data pipeline. Do not use for actual legal decisions.
- Base model license: Apache 2.0.
Training details
- Base: Qwen/Qwen3-4B (Apache-2.0)
- Method: supervised fine-tuning (SFTTrainer, TRL 1.13)
- Optimizer: AdamW 8-bit (bitsandbytes), lr=2e-5, cosine schedule, 100 warmup steps
- Epochs: 3, batch size 4, gradient accumulation 4 (effective batch 16)
- Gradient checkpointing: yes
- Hardware: RunPod L40S (48 GB)
- Data SHA-256:
53697c5762c011b57cf80e00d9ddf6aa5cb54eae948eca8b59c569d7123ab440