Case-law block tagger
A token-classification model that labels the structural components of a U.S. case-law opinion: given the OCR'd text of an opinion as lightly-marked-up HTML, it tags the caption and headmatter elements and the opinion skeleton as character spans.
Classes (12, BIO): party, separator, docketnumber, court,
attorneys, judges, datefiled, otherdate, history,
disposition, author, heading. Everything else (body prose) is
O.
Built by the Free Law Project for structuring scanned case-law reporters; also applicable to born-digital opinion text serialized to the same input format.
Model details
- Architecture: ModernBERT-large (0.4B), fine-tuned from CaseLawModernBERT-large (continued-pretrained on 13B words of U.S. court opinions), 8,192-token context.
- Input format: one court case per sequence, as minimal HTML —
<p>per text block, plus<blockquote>,<em>,<sup>; footnote content omitted. One\nbetween blocks. The model never generates text; it emits per-token BIO labels (12 classes × begin/inside + outside = 25 label ids). - Training data: ~4,400 pages of scanned U.S. reporter volumes
(16 volumes across 8 reporter series), OCR'd through a
multi-engine pipeline; labels human-annotated (a ~680-page golden
set) and model-seeded + human-reviewed (a ~3,900-page extension).
Augmentations: random block-boundary drops (OCR segmentation
robustness) and styling dropout (
<em>/<sup>-stripped copies). - Training: warm-started from a golden-set fine-tune; lr 3e-5, effective batch 16, bf16, best epoch by span-F1 on a mixed validation set, early stopping.
Evaluation
Span-level F1: exact (start, end, label) sets, gold and predictions normalized identically (whitespace/markup edge anchoring; the "normalized" column additionally ignores edge punctuation). No prediction post-processing. All evaluation volumes are unseen in training.
| eval set | span-F1 strict | normalized |
|---|---|---|
| validation (3 volumes, human-reviewed gold) | 0.929 | 0.935 |
| test (4 volumes, human-reviewed gold) | 0.937 | 0.950 |
Generalization detail: on the validation volume from a reporter series entirely absent from training (Pacific 3d), span-F1 is 0.934. On the test set, the in-sample-reporter volume scores 0.979; the fully out-of-sample reporter series (South Eastern 2d, Bankruptcy Reporter) score 0.872–0.903.
Per-class on validation (strict): party 0.990, attorneys 0.983, datefiled 0.981, author 0.968, court 0.963, separator 0.994, docketnumber 0.949, heading 0.938, judges 0.865, otherdate 0.842, history 0.815, disposition 0.801.
Caption classes (the six fields composing a case caption — party, separator, docketnumber, court, datefiled, otherdate — typically the primary extraction targets), aggregated as micro-F1 over the six:
| eval set | strict | normalized | gold spans |
|---|---|---|---|
| validation | 0.972 | 0.973 | 627 |
| test | 0.979 | 0.986 | 2,005 |
Per class (strict, val / test): party 0.990 / 0.977 · separator 0.994 / 0.967 · docketnumber 0.949 / 0.968 · court 0.963 / 0.997 · datefiled 0.981 / 0.998 · otherdate 0.842 / 0.750 (rare secondary argued/decided dates; 12–28 gold spans per set).
Usage
from transformers import (AutoModelForTokenClassification,
AutoTokenizer)
repo = "freelawproject/caselaw-block-tagger" # e.g. freelawproject/caselaw-block-tagger
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo)
text = (
"<p>Jane ROE, Appellant,</p>\n"
"<p>v.</p>\n"
"<p>STATE of Example, Appellee.</p>\n"
"<p>No. 24-123</p>\n"
"<p>Court of Appeals of Example.</p>\n"
"<p>January 1, 2026</p>\n"
"<p>Affirmed.</p>"
)
enc = tok(text, return_tensors="pt", return_offsets_mapping=True)
logits = model(**{k: v for k, v in enc.items()
if k != "offset_mapping"}).logits
labels = [model.config.id2label[i]
for i in logits.argmax(-1)[0].tolist()]
Decode BIO runs over the offset mapping to recover character spans. For best results segment the input at case boundaries (one case per sequence, ≤8k tokens) and strip footnote content — the model was trained that way.
Limitations
- Trained on U.S. reporter typography; other layout traditions (official state reporters, foreign case law) are untested.
dispositionis the hardest class (two-tier convention: a final short ruling plus in-text holdings; 0.77 on test), andjudgesdegrades on unfamiliar concur-line formats (0.58 on out-of-sample reporters);otherdatehas thin support.- Citations are deliberately NOT a class — they are handled by a separate model.
- Input must follow the canonical serialization above; markup tokens are label-masked during training, so unexpected markup degrades output.