legal-colbert-extractor
A small, open late-interaction (ColBERT) retriever optimized for extracting clauses from real, filed commercial contracts — assignment, termination (and its subtypes: for cause, for convenience, upon change of control…), notices, cure periods, indemnification, confidentiality, limitation of liability, governing law, and ~50 other provision subtypes. 149M parameters (ModernBERT-base backbone), 128-d token vectors, MaxSim scoring. Runs comfortably on CPU.
This is the real-contract sibling of legal-colbert-clause-retriever. The two share a lineage but are selected on different gates: the clause-retriever maximizes the public MLEB template-clause benchmark; this model maximizes retrieval quality on SEC-filed contract text with human-adjudicated relevance labels. If you are pulling clauses out of actual contracts, use this one; if you are matching against idealized clause-library text, use the sibling.
Results
Primary gate — a blind evaluation over 78 fresh SEC EDGAR Exhibit-10 contracts (830 passages, 15 clause-type queries, 171 relevance labels adjudicated independently three ways; zero overlap with any training data):
| Model | Adjudicated blind NDCG@10 | 58-subtype discrimination |
|---|---|---|
| legal-colbert-extractor | 0.7781 | 0.6968 |
| legal-colbert-clause-retriever (sibling) | 0.7683 | 0.6911 |
| earlier best (P6b) | 0.7642 | 0.6510 |
On MLEB Contractual Clause Retrieval it scores 0.8298 NDCG@10 (the sibling scores 0.8374 — that benchmark is the sibling's selection gate, not this model's).
Query phrasing matters
Trained on rigid clause-type queries and evaluated with descriptive definitions. Phrase queries as full descriptions of the provision you want:
- ✅
"Find contractual provisions governing termination rights, termination for cause, or the consequences of ending the agreement." - ✅
"Find provisions related to 'Termination for Convenience'." - ⚠️
"termination clause"— bare keywords underperform.
Queries are truncated at 48 tokens; documents at 300 tokens (chunk accordingly, ~180–220 words).
Extraction pattern (many contracts)
Index all contract chunks once, run one definition query per clause type, then group hits by document and take top-k within each document — for extraction you want the best candidate in every contract, not a global leaderboard.
from pylate import models
model = models.ColBERT("kmad00/legal-colbert-extractor")
q = model.encode(["Find provisions related to 'Termination for Cause'."], is_query=True)
d = model.encode(chunks, is_query=False) # your contract chunks
Training
Continuation fine-tune lineage from lightonai/GTE-ModernColBERT-v1 through the clause-retriever line, then two extractor-specific rounds: (1) 62 clause subtypes mined from the full LEDGAR corpus (SEC EDGAR provisions with section-heading labels) plus FAR Part 52 public-domain clauses, (2) within-family and model-mined hard negatives (sibling-subtype confusables the incumbent model actually mis-ranked), contrastive training in very light doses (≤300 steps, lr ≤1e-7) with a CUAD anchor. Selection required improvement on the adjudicated blind gate with no MLEB regression below 0.82.
Limitations
- English-language, US-style commercial contracts.
- Multi-vector storage is heavier per document than single-vector embedders.
- Slightly weaker than the sibling on idealized template-clause text (that's the trade).
- Clause types far outside the training taxonomy get lower confidence.
License
CC BY 4.0. Training data: CUAD (CC BY 4.0), LEDGAR (SEC EDGAR public filings), ACORD (CC BY 4.0), FAR Part 52 (US public domain).