dinghy-law-0.6b

A 0.6B-parameter text-embedding model for legal retrieval, fine-tuned from Qwen/Qwen3-Embedding-0.6B for statute, case-law, and contract text. It is intended for retrieval, reranking, and semantic search over legal corpora.

On MTEB(Law, v1) it scores 65.88 by Mean(Task) nDCG@10. In our reproduction of the benchmark, the only models that score higher are 7B+ parameters — 12 to 23 times its size — or closed commercial APIs. That places it among the strongest open models under 7B parameters, and, to our knowledge, the strongest under 1B by roughly seven points. It reaches about 94% of the top model's score at under 8% of its parameter count — small enough to run on a CPU with no GPU (see Intended use and positioning). These are our own measurements against published scores, not an official leaderboard placement (see Evaluation protocol).

Results

MTEB(Law, v1)

Mean nDCG@10 (Mean(Task)) over the eight MTEB(Law) tasks. This model is measured with a leaderboard-faithful harness (see Evaluation protocol); scores for the other models are their published MTEB(Law) results. The ordering below reflects those numbers, not an official leaderboard submission.

• In this evaluation it scores above every model up to 14B parameters except one 8B fine-tune, and above the strongest commercial legal embedding model on the benchmark (voyage-law-2).
• Among the models under 1B parameters it is the highest we measured by a wide margin: the next-best sub-1B model scores 58.96 (Snowflake arctic-embed-l-v2.0) and the same-size F2LLM-v2-0.6B scores 57.86 — a gap of about seven points.
• The backbone is unchanged from Qwen3-Embedding-0.6B. On the seven tasks where the base can be evaluated at full context, the untuned base scores 61.7 and this model 64.8; the adaptation described below accounts for the gain.

Intended use and positioning

Strong legal retrieval has generally meant large models (7–14B) or a commercial API. Those can be self-hosted, and many teams do — but it takes real hardware, so the practical barrier is cost and infrastructure. dinghy-law-0.6b reaches near the top of the legal benchmark at 0.6B parameters, which lowers that barrier rather than removing an impossibility: it runs on a commodity CPU with no GPU, so competitive legal-domain retrieval is accessible to more deployments — on-premise, air-gapped, or on modest hardware — not only to teams that can host a large model or call an API.

Typical uses: first-stage retrieval and reranking for legal RAG, statute and precedent search, clause and contract matching, and clustering or deduplication of legal text. It produces a single dense vector per input; it is a retrieval model, not a generative model, and does not produce legal advice.

Method

The model is a contrastive fine-tune of Qwen3-Embedding-0.6B, specialized for legal retrieval and then merged back toward the base to preserve general capability.

1. A strong instruction-following base. Qwen3-Embedding-0.6B is an LLM-derived embedding model that already accepts a task instruction on the query side [Qwen3-Embedding, 2506.05176]. Untuned it is a competent general retriever but well down the legal benchmark, so the adaptation only has to add legal specificity, not general retrieval competence.

2. Contrastive domain fine-tuning. The model is fine-tuned with an InfoNCE objective [1807.03748] and in-batch negatives [DPR, 2004.04906] on query→passage pairs drawn from public legal corpora: German case law (GerDaLIR), European Court of Human Rights and U.S. Supreme Court opinions, Chinese legal cases, and public-domain Indian statutes (the Indian Penal Code, the Constitution of India, and 880+ central Acts from indiacode.nic.in).

3. Instruction conditioning. Queries carry a task-specific instruction, matching the base model's native interface and the instruction-tuned-embedding line of work [INSTRUCTOR, 2212.09741]. The document side is encoded without an instruction.

4. Gradient-aligned hard-negative mining. Rather than mining nearest-embedding negatives — the standard approach [ANCE, 2007.00808] — candidate negatives are selected by whether the gradient of the hard-negative-augmented InfoNCE loss aligns with the true retrieval gap. In the legal setting the nearest-embedding negative is frequently a false negative [cf. RocketQA, 2010.08191] and, empirically, is worse than a random negative; gradient-aligned selection avoids that failure. Negatives are mined in the base model's own representation geometry, so they are specific to this backbone.

5. Statute-coverage breadth. The Indian-statute portion was deliberately broadened beyond the Indian Penal Code to constitutional, procedural, and commercial statute language. Broadening the coverage lifts performance across tasks, not only statute identification, because the encoder adapts to a wider range of legal register.

6. Weight-space merge-back. The fine-tuned weights are interpolated back toward the base, θ = (1 − α)·θ_base + α·θ_finetuned, at α = 0.2 [WiSE-FT, 2109.01903; Model Soups, 2203.05482]. A small model fine-tuned hard on one domain forgets general retrieval; the merge recovers that general capability in weight space, in a single step and with no retraining, while keeping the legal gains. The α that maximizes MTEB(Law) is low (0.2), which is why the general-domain trade above is small: most of the base's weights are retained.

The specialization uses standard contrastive methods; the merge-back is what lets a 0.6B model keep both its legal gains and its general retrieval competence, which a hard domain fine-tune would otherwise lose.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Hanno-Labs/dinghy-law-0.6b-v1")

query = "What is the punishment for criminal breach of trust by a public servant?"
docs = [
    "Whoever, being in any manner entrusted with property ... commits criminal breach of trust ...",
    "A contract of sale is a contract whereby the seller transfers ... the property in goods ...",
]

# The legal retrieval instruction is applied to queries and omitted from documents.
q_emb = model.encode_query(query)
d_emb = model.encode_document(docs)

scores = model.similarity(q_emb, d_emb)  # cosine
print(scores)

The query-side instruction is stored in the model config (default_prompt_name), so encode_query applies it and encode_document omits it; a bare model.encode(text) also applies the query instruction by default. The model emits 1024-dimensional embeddings, uses cosine similarity, and encodes sequences up to 8192 tokens by default (extensible to the base model's 32k context via model.max_seq_length).

Loading and serving. These results, and the exact embeddings, come from loading the model through sentence-transformers, which applies the model's final projection head. Runtimes that pool and normalize without a post-pooling projection — vLLM, Text Embeddings Inference, and GGUF/llama.cpp — omit that head; their embeddings agree with the reference to about 0.997 cosine (a small, roughly 0.3-point retrieval difference on MTEB(Law)). A projection-free build for those runtimes is planned.

Training data

All training sources are permissively licensed or public-domain legal text; each retains its original license (public domain, CC0, MIT, Apache-2.0, or CC-BY-4.0), and all permit redistribution and commercial use. The training data is released as two datasets — legal-retrieval-pairs-v1 and legal-hardnegs-qwen3-emb-0.6b-v1 — whose cards carry a per-source license and attribution table; CC-BY-4.0 material (ECtHR, SCOTUS, LEDGAR) is used with attribution. The Indian-statute component is built from public-domain enactments (Indian Copyright Act, 1957, §52(1)(q)); a cleaned slice is also released as indian-ipc-statute-identification under CC-BY-4.0. Compilation, curation, and hard-negative mining are © 2026 Clause Logic Inc.

Evaluation protocol

This model's MTEB(Law) number is produced with a leaderboard-faithful harness: query-side instruction only, raw documents, EOS token appended, max sequence length 8192, bf16, cosine similarity — matching the recipe the MTEB leaderboard uses for instruction embedders. The harness is validated against the board by reproducing a reference model's published score to within ±0.002. Scores quoted for other models are their published MTEB(Law) results; comparing our harness measurement against those published numbers is a reproduction, not an official leaderboard submission.

Limitations

• The residual gap to the top of MTEB(Law) is concentrated in German-law case retrieval and long-document summarization, where a 0.6B backbone has limited capacity; more in-domain data does not close it. Tasks needing that headroom are better served by a larger backbone.
• General-domain (non-legal) retrieval carries a small cost relative to the base, most visible on clustering tasks; the low merge weight (α = 0.2) keeps it bounded.
• The model is multilingual only to the extent of its training languages (English, German, Chinese legal text); other languages and jurisdictions are out of distribution.
• It is a retrieval/embedding model. It does not generate text and does not constitute legal advice.

License and attribution

Released under Apache-2.0. Copyright © 2026 Clause Logic Inc. The base model, Qwen3-Embedding-0.6B, is licensed by its authors under Apache-2.0; this fine-tune does not relicense it.

Citation

@misc{dinghy-law-0.6b,
  title  = {dinghy-law-0.6b: a compact legal text-embedding model},
  author = {Solka, Stephen},
  year   = {2026},
  note   = {Hanno Labs / Clause Logic Inc.},
  howpublished = {\url{https://huggingface.co/Hanno-Labs/dinghy-law-0.6b-v1}}
}

References

• Qwen3 Embedding — arXiv:2506.05176
• WiSE-FT: Robust fine-tuning of zero-shot models — arXiv:2109.01903
• Model soups — arXiv:2203.05482
• Representation Learning with Contrastive Predictive Coding (InfoNCE) — arXiv:1807.03748
• Dense Passage Retrieval (DPR) — arXiv:2004.04906
• ANCE: Approximate Nearest Neighbor Negative Contrastive Learning — arXiv:2007.00808
• RocketQA (false-negative denoising) — arXiv:2010.08191
• INSTRUCTOR: instruction-finetuned embeddings — arXiv:2212.09741
• MTEB — arXiv:2210.07316