sec-bert-finer-ord-ner
A token-classification (NER) fine-tune of
nlpaueb/sec-bert-base on the
FiNER-ORD dataset,
for extracting person (PER), location (LOC), and organization (ORG)
entities from financial/SEC-filing-style English text.
This is an academic exercise: a small pipeline that runs financial-news NLP (sentiment + NER) over a corpus of company news articles.
Intended use
- Named entity recognition (PER/LOC/ORG) on financial news, earnings-call transcripts, and SEC-filing-style English text.
- Non-commercial use only — see License below.
Not intended for: general-domain NER (it's tuned for financial text), languages other than English, or any production/commercial deployment.
Label scheme
BIO tagging over three entity types:
| id | label |
|---|---|
| 0 | O |
| 1 | B-PER |
| 2 | I-PER |
| 3 | B-LOC |
| 4 | I-LOC |
| 5 | B-ORG |
| 6 | I-ORG |
Training data
FiNER-ORD
(gtfintechlab/finer-ord), a manually annotated financial NER dataset.
Token-level rows were regrouped into sentences (grouped by doc_idx/sent_idx)
and labels aligned to WordPiece subwords, keeping only the first subword of
each token labeled (other subwords set to -100, ignored in the loss).
| split | sentences |
|---|---|
| train | 3,262 |
| validation | 402 |
| test | 1,075 |
Training procedure
Fine-tuned from nlpaueb/sec-bert-base with Hugging Face Trainer:
- learning rate: 3e-5
- batch size: 16 (train) / 32 (eval)
- epochs: 8
- weight decay: 0.01
- mixed precision (fp16)
load_best_model_at_end=True, selected by validation F1- seed: 42
Evaluation results
Metrics on the FiNER-ORD test split, using the best-validation-F1 checkpoint (selected during training):
| metric | value |
|---|---|
| F1 | 0.770 |
| Precision | 0.741 |
| Recall | 0.802 |
| Accuracy | 0.979 |
(Best validation-split F1 during training: 0.820.)
Downstream pipeline evaluation (real financial news, LLM-judge)
The numbers above are the FiNER-ORD test split. Separately, this checkpoint
is run in production inside
portfolio-nlp — a pipeline that
chunks full financial-news articles (not FiNER-ORD's short annotated
sentences), runs this model over every chunk, and merges per-chunk BIO
predictions back into article-level entity spans (see "Recommended
pre/post-processing" below). That pipeline's accuracy is tracked separately
via an LLM-as-judge eval (docs/evaluation.md in the repo above), most
recently at n=8000 real articles (2026-09-12, post word-boundary-merge fix):
| metric | value |
|---|---|
| micro F1 | 0.858 |
| micro precision | 0.840 |
| micro recall | 0.876 |
| hallucination rate¹ | 0.160 |
| F1 — ORG | 0.812 |
| F1 — LOC | 0.881 |
| F1 — PER | 0.926 |
¹ share of predicted spans an LLM judge, shown the full source article, couldn't confirm as a real entity.
These are not a substitute for the FiNER-ORD test-split numbers above — different data (real news vs. held-out annotated sentences), different scoring (LLM judge vs. exact-match seqeval), and they depend on the downstream pipeline's chunking/merge logic, not just this checkpoint in isolation. Included as a second, real-world-traffic data point.
Recommended pre/post-processing
Chunk long documents; don't truncate. This model's max sequence length is BERT-base-sized (512 tokens), well short of a typical news article or filing. Split on sentence/paragraph boundaries, run each chunk separately, and re-offset each chunk's character spans back into full-document coordinates when merging — truncating instead silently drops every entity past the cutoff.
If you aggregate raw per-token BIO predictions yourself (rather than
transformers' built-in aggregation_strategy), merge spans word-boundary-
aware: only a word's first WordPiece subword should open, close, or
redirect a span; a continuation subword (same word) should only ever extend
whatever its word's first subword already decided, never be read
independently. This matches how the model was trained — per "Training
data" above, only each word's first subword carries a label; continuation
subwords get -100 and zero training signal, so their raw argmax is
unreliable on its own. A production deployment of this model that merged by
raw per-token argmax (subword-naive) produced real, systematic artifacts:
bare-digit spans like "3" tagged ORG, split off "3M" — 1,859 such
"3"-only entities in one 17.6M-row NER table, 62% of a wider single-digit-
ORG pattern, plus the same shape off "3Q24", "1MDB", "L3Harris", and
"Boeing's 777 jet". If using the transformers pipeline, prefer
aggregation_strategy="first" over "simple"/"average" — "first" uses
each word's first-subword label, the same rule the model was trained under;
the other strategies can let an undertrained continuation subword's score
pull a span's boundary or label. A defensive minimum-entity-length filter
(reject 1-character spans) is a reasonable last-resort net regardless of
which merge approach you use.
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
tokenizer = AutoTokenizer.from_pretrained("gamug/sec-bert-finer-ord-ner")
model = AutoModelForTokenClassification.from_pretrained("gamug/sec-bert-finer-ord-ner")
# aggregation_strategy="first" matches how this model was trained (only each
# word's first WordPiece subword carries a label) — see "Recommended
# pre/post-processing" above.
ner = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="first")
ner("3M Company (NYSE:MMM) reported results with CEO Mike Roman on the call.")
License
This model is a derivative of two upstream works with different licenses:
- Base model
nlpaueb/sec-bert-base:CC-BY-SA-4.0 - Training data
gtfintechlab/finer-ord:CC-BY-NC-4.0
Because the training data is non-commercial-only, this fine-tuned model is
released under CC-BY-NC-4.0: attribution required, non-commercial use
only. Please credit both upstream works (linked above) if you use or build
on this model.