Polygraf NER Stage 3 First Model
This is the first English NER model produced for Stage 3 of the Polygraf
Applied NLP / NER technical task. It fine-tunes
distilbert/distilbert-base-cased on the manually reviewed Stage 2 dataset.
Labels
The model predicts BIO tags for eight entity types:
PERSONORGANIZATIONLOCATIONTIMEDATEPRODUCTWORKOFARTJOBAMOUNT
The exact id2label and label2id mappings are stored in config.json.
Training data
- Dataset:
Hasanblanka/polygraf-ner-stage_2-dataset - Dataset revision:
bce1f32778a333297ba1bf9749c13dd394e6256d - Train records: 506
- Test records: 127
- Base model revision:
6ea81172465e8b0ad3fddeed32b986cdcdcffcf0
No validation split was created. The held-out test split was not used for early stopping or hyperparameter selection.
Training configuration
- Epochs: 4
- Batch size: 8
- Gradient accumulation: 2
- Effective batch size: 16
- Optimizer: AdamW
- Learning rate:
5e-5 - Weight decay:
0.01 - Warmup ratio:
0.1 - Seed: 42
- Maximum sequence length: 512
- Truncation: disabled
- Device used for this run: CPU
Evaluation
| Metric | Train | Test |
|---|---|---|
| Precision | 0.7455 | 0.5473 |
| Recall | 0.7713 | 0.6260 |
| Entity F1 | 0.7582 | 0.5840 |
| Macro F1 | 0.6735 | 0.5348 |
| Token accuracy | 0.9635 | 0.9201 |
Test results by entity type:
| Label | Precision | Recall | F1 |
|---|---|---|---|
| PERSON | 0.6071 | 0.7816 | 0.6834 |
| ORGANIZATION | 0.4795 | 0.5785 | 0.5243 |
| LOCATION | 0.6398 | 0.8306 | 0.7228 |
| TIMEDATE | 0.7302 | 0.6434 | 0.6840 |
| PRODUCT | 0.0938 | 0.0732 | 0.0822 |
| WORKOFART | 0.4136 | 0.7165 | 0.5245 |
| JOB | 0.5306 | 0.3662 | 0.4333 |
| AMOUNT | 0.7358 | 0.5417 | 0.6240 |
The metrics/ directory contains the complete machine-readable results,
benchmark, manual wild-test metrics, and test error breakdown.
Resource measurements
- Parameters: 65,203,985
- Saved model directory: approximately 261.5 MB
- CPU throughput at batch size 1: 19.15 sequences/s
- CPU token throughput: 1,792.72 tokens/s
- Peak process RAM: approximately 639 MB
- Incremental peak RAM: approximately 244 MB
The benchmark measured 100 samples in a fresh process on an Intel Core i5-13420H CPU. Results will vary by hardware and input length.
Usage
from transformers import pipeline
model_id = "Hasanblanka/polygraf-ner-stage3-first-model"
ner = pipeline(
task="token-classification",
model=model_id,
tokenizer=model_id,
aggregation_strategy="simple",
)
text = "Maya Ortiz joined Northstar Robotics in Lisbon on March 12, 2026."
for entity in ner(text):
print(entity)
Wild-sample QA
Ten fresh, manually annotated samples outside the main dataset were used as a
targeted stress test. Entity F1 was 0.5376. Observed problems included
excessive WORKOFART predictions, missed PRODUCT and JOB entities,
organization/location confusion, organization/job confusion, fragmented
multiword entities, subword fragmentation, and boundary errors.
This small manual set is useful for diagnosis but is not a statistically representative benchmark.
Limitations
- This is a first baseline and is not suitable for production.
PRODUCTrecognition is particularly weak.WORKOFARTis overpredicted.- News-heavy training data limits domain coverage.
- The project uses a custom flat, non-overlapping annotation policy.
- Nested entities are not supported.
License and data terms
The base DistilBERT checkpoint is Apache-2.0. The fine-tuning dataset combines
sources with different terms, including evaluation-use-only starter data and
CC BY / CC BY-SA Open Newswire records. This model is therefore published with
license: other and is intended for evaluation and research. Users must
review and follow the source and licensing notes in the Stage 2 dataset card.