Historical NER Baseline

This model is a historical named entity recognition model trained on HIPE-style historical newspaper data.

It is based on dbmdz/bert-base-historic-multilingual-cased and fine-tuned for token classification using the NE-COARSE-LIT annotation column.

The model predicts coarse named entity labels for historical French text.

Model description

This is the baseline model from the historical-ner experiments.

Label normalization

The original data contained labels from slightly different annotation schemes. Labels were normalized before training.

Examples:

B-PER       -> B-pers
I-PER       -> I-pers
B-LOC       -> B-loc
I-ORG       -> I-org
B-STREET    -> B-loc
B-BUILDING  -> B-loc
B-HumanProd -> B-prod
B-object    -> B-prod
B-work      -> B-prod
B-date      -> B-time

Evaluation

Evaluation was performed on:

data/hipe2020/fr/HIPE-2022-v2.1-hipe2020-test-fr.tsv

Overall results

Per-class results

How to use

Install the required packages:

pip install torch transformers

Then run:

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_id = "emanuelaboros/historical-ner-baseline"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

ner = pipeline(
    "token-classification",
    model=model,
    tokenizer=tokenizer,
    aggregation_strategy="simple",
)

text = "Charlotte née Bourgoin, femme de Joseph Digiez, fut admise par le Conseil."

predictions = ner(text)

for entity in predictions:
    print(entity)

Example output format:

[
    {
        "entity_group": "pers",
        "score": 0.98,
        "word": "Charlotte née Bourgoin",
        "start": 0,
        "end": 24,
    },
    {
        "entity_group": "pers",
        "score": 0.97,
        "word": "Joseph Digiez",
        "start": 35,
        "end": 48,
    },
    {
        "entity_group": "org",
        "score": 0.91,
        "word": "Conseil",
        "start": 70,
        "end": 77,
    },
]

Related models

• emanuelaboros/historical-ner-baseline
• emanuelaboros/historical-ner-stacked
• emanuelaboros/historical-ner-time