SpanMarker with bert-base-cased on FewNERD, CoNLL2003, and OntoNotes v5
This is a SpanMarker model trained on the FewNERD, CoNLL2003, and OntoNotes v5 dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-cased as the underlying encoder.
Model Details
Model Description
Model Sources
Model Labels
| Label | Examples |
|---|
| ORG | "Texas Chicken", "IAEA", "Church 's Chicken" |
Evaluation
Metrics
| Label | Precision | Recall | F1 |
|---|
| all | 0.7958 | 0.7936 | 0.7947 |
| ORG | 0.7958 | 0.7936 | 0.7947 |
Uses
Direct Use for Inference
from span_marker import SpanMarkerModel
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-orgs")
# Run inference
entities = model.predict("Postponed: East Fife v Clydebank, St Johnstone v")
Downstream Use
You can finetune this model on your own dataset.
from span_marker import SpanMarkerModel, Trainer
# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-orgs")
# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003
# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
model=model,
train_dataset=dataset["train"],
eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("tomaarsen/span-marker-bert-base-orgs-finetuned")
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|
| Sentence length | 1 | 23.5706 | 263 |
| Entities per sentence | 0 | 0.7865 | 39 |
Training Hyperparameters
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 3
Training Results
| Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy |
|---|
| 0.7131 | 3000 | 0.0061 | 0.7978 | 0.7830 | 0.7904 | 0.9764 |
| 1.4262 | 6000 | 0.0059 | 0.8170 | 0.7843 | 0.8004 | 0.9774 |
| 2.1393 | 9000 | 0.0061 | 0.8221 | 0.7938 | 0.8077 | 0.9772 |
| 2.8524 | 12000 | 0.0062 | 0.8211 | 0.8003 | 0.8106 | 0.9780 |
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Carbon Emitted: 0.248 kg of CO2
- Hours Used: 1.766 hours
Training Hardware
- On Cloud: No
- GPU Model: 1 x NVIDIA GeForce RTX 3090
- CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
- RAM Size: 31.78 GB
Framework Versions
- Python: 3.9.16
- SpanMarker: 1.5.1.dev
- Transformers: 4.30.0
- PyTorch: 2.0.1+cu118
- Datasets: 2.14.0
- Tokenizers: 0.13.3
Citation
BibTeX
@software{Aarsen_SpanMarker,
author = {Aarsen, Tom},
license = {Apache-2.0},
title = {{SpanMarker for Named Entity Recognition}},
url = {https://github.com/tomaarsen/SpanMarkerNER}
}