deka-qwen3-emb-0.6b-infonce-fact-anompos
Full fine-tune of Qwen/Qwen3-Embedding-0.6B for Thai Supreme Court (deka) case retrieval.
- objective:
infonce - document register:
fact(query side trained againstfacttext) - best epoch: 5 (early-stopped on dev R@10 = 33.42)
- pooling: last token (left padding) + L2 normalize
- lr 2e-05, batch 16, tau 0.05, max_len doc 1300 / query 256, seed 42
Test results (full corpus, clean qrels)
| eval register | R@5 | R@10 | R@50 | R@100 | ndcg@10 | mrr |
|---|---|---|---|---|---|---|
| user | 6.67 | 10.48 | 30.48 | 43.81 | 6.35 | 0.0856 |
| lawer | 11.9 | 21.43 | 47.14 | 59.52 | 13.55 | 0.1567 |
| fact | 20.0 | 29.05 | 60.0 | 75.71 | 20.02 | 0.2365 |
Usage
Queries must carry the instruction prefix used in training; documents must not.
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("anonymousresearch123/deka-qwen3-emb-0.6b-infonce-fact-anompos")
q = m.encode(["ลูกจ้างถูกเลิกจ้างโดยไม่บอกกล่าวล่วงหน้า"], prompt_name="query")
d = m.encode(["<คำพิพากษาฎีกา ...>"])
print(m.similarity(q, d))
Plain transformers:
import torch, torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer
tok = AutoTokenizer.from_pretrained("anonymousresearch123/deka-qwen3-emb-0.6b-infonce-fact-anompos", padding_side="left")
net = AutoModel.from_pretrained("anonymousresearch123/deka-qwen3-emb-0.6b-infonce-fact-anompos", torch_dtype=torch.bfloat16).eval()
INSTRUCT = "Given a legal case search query, retrieve relevant prior Supreme Court cases"
texts = [f"Instruct: {INSTRUCT}\nQuery: ลูกจ้างถูกเลิกจ้าง..."] # query side only
enc = tok(texts, padding=True, truncation=True, max_length=1300, return_tensors="pt")
with torch.no_grad():
h = net(**enc).last_hidden_state
emb = F.normalize(h[:, -1].float(), p=2, dim=-1)
Left padding is required — pooling takes the last position.
Training data
anonymousresearch123/deka-retrieval — splits train / eval / test.
training_history.json in this repo holds the full per-epoch log and config.