LegalIR YuITC v2
Experimental Vietnamese legal retrieval model fine-tuned from
YuITC/vietnamese-embedding-vn-legal
for the UIT Data Science Challenge 2026 LegalIR task.
This upload preserves a completed experiment for reproducibility. It was not promoted as the project's default retriever, because its standalone and fused validation results did not beat the locked frozen-model candidate-pool gate.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("QuanDuong184/legalir-yuitc-v2")
texts = [
"Câu hỏi pháp luật bằng tiếng Việt",
"Đoạn văn bản pháp luật cần so khớp",
]
embeddings = model.encode(texts, normalize_embeddings=True)
scores = embeddings @ embeddings.T
The model uses:
- maximum sequence length: 512 tokens;
- output dimension: 1024;
- CLS pooling;
- L2-normalized embeddings;
- cosine similarity;
- no query or document prefix.
For document retrieval in the associated experiment, every chunk was encoded independently and each document received the maximum cosine similarity over its chunks.
Training
- Training scope: queries whose
document_group_fold != 0. - Validation scope: held-out
document_group_fold = 0. - Training rows: 6,001 from 5,513 unique queries.
- Each row: one positive and seven explicit negatives.
- Loss: packed-candidate multiple-negatives ranking loss.
- Temperature: 0.02.
- Epochs: 1 (751 optimizer steps).
- Learning rate: 1e-5.
- Effective batch construction: device batch size 1 with 8 gradient accumulation steps.
- Precision: BF16.
- Final training loss: 0.505406.
The split and training source hashes are recorded in the associated project artifacts. Training did not include fold-0 queries.
Evaluation
Held-out fold 0 contains 1,400 queries and 1,535 gold query-document pairs. Rankings were produced over 244,612 chunks using maximum chunk score per document, with the top 200 documents retained per query.
| Cutoff | Macro recall |
|---|---|
| 1 | 0.57976 |
| 2 | 0.72899 |
| 3 | 0.80482 |
| 5 | 0.86542 |
| 20 | 0.94952 |
| 50 | 0.97232 |
| 100 | 0.98565 |
| 170 | 0.98946 |
When fused with the locked three-branch control using document-level weighted
RRF (k=60), the best diagnostic weight in {0.25, 0.50, 0.75, 1.00, 1.25}
was 1.25:
| Cutoff | Control | Control + this model |
|---|---|---|
| 5 | 0.85952 | 0.88393 |
| 20 | 0.96107 | 0.97256 |
| 50 | 0.98310 | 0.98845 |
| 100 | 0.99155 | 0.99393 |
| 170 | 0.99500 | 0.99500 |
The fusion weight was selected on the same fold and is therefore diagnostic, not an independently validated hyperparameter. At cutoff 170, the fusion added two gold pairs and lost two gold pairs relative to the control.
Limitations
- Intended for Vietnamese legal information retrieval; performance outside this domain has not been established.
- The fine-tuning dataset comes from a competition task and may not represent all Vietnamese legal language or current law.
- The task-tuned candidate and frozen control were ranked using different chunk corpora. Document-level fusion is valid, but differences cannot be attributed solely to the model.
- Embedding similarity is not legal advice and should not be used as the sole basis for legal decisions.
Reproducibility
- Training code and audit: https://github.com/Quanduong-184/LegalIR-YuITC
- Result-recording commit:
f16b1f9 - Fusion-audit commit:
742a2c3 model.safetensorsSHA-256:be92151c789f18e778700088f639d168d21c8c49374b5adbfc3251244f946d3f