LeBSE — Legal-domain Sentence Embeddings (a legal-adapted LaBSE)
LeBSE-v2 is LaBSE fine-tuned with
citation supervision on U.S. case law, so legally-related opinions land closer together. It is a
drop-in sentence-transformers model (768-dim, same tokenizer, keeps LaBSE's multilingual base).
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("ahbond/lebse")
emb = m.encode(["The district court lacked subject-matter jurisdiction over the claim."],
normalize_embeddings=True)
How it was trained
SPECTER-style contrastive fine-tuning: positive pairs are (citing opinion body, cited opinion body)
from the CourtListener citation graph (100,344 pairs); the rest of
the batch are negatives (MultipleNegativesRankingLoss). One NVIDIA A10, batch 96, max_seq_length
128, 2 epochs. No case outcomes are used — the signal is citation relatedness only.
Evaluation (held out, opinion-disjoint)
| eval | base LaBSE | LeBSE-v2 | Δ AUROC (95% CI) |
|---|---|---|---|
| citation retrieval (trained relation) | 0.765 | 0.971 | +0.206 [+0.190, +0.223] |
| docket-lineage (independent relation, unseen) | 0.545 | 0.562 | +0.018 [+0.004, +0.031] |
LeBSE-v2 dramatically improves the relatedness it was trained on and transfers a small-but-significant amount to an independent legal relation (a district opinion ↔ its appellate reviewer, matched by docket number, never trained on). It also improves embedding isotropy (anisotropy 0.570 → 0.259).
An earlier v1 used unsupervised SimCSE and did not beat base LaBSE — see the repo for that honest negative result. This model is v2.
Intended use & limits
Legal opinion/paragraph retrieval, citation recommendation, clustering, and as a frozen legal-domain feature extractor. Not for legal advice or case outcome decisions. U.S. federal law only; specialized to citation-type relatedness; encodes a ~128-token paragraph, not a whole opinion.
Code, training, and evaluation: https://github.com/ahb-sjsu/lebse · License: Apache-2.0 (training data is U.S. federal case law, public domain).