Polygraf NER Stage 4 Final Model
This is the final improved English NER model produced for Stage 4 of the
Polygraf Applied NLP / NER technical task. It fine-tunes
distilbert/distilbert-base-cased on the reviewed and expanded Stage 4
dataset.
Labels
The model predicts BIO tags for eight flat, non-overlapping entity types:
PERSONORGANIZATIONLOCATIONTIMEDATEPRODUCTWORKOFARTJOBAMOUNT
The exact id2label and label2id mappings are stored in config.json.
Architecture
- Architecture:
DistilBertForTokenClassification - Parameters: 65,203,985
- Hidden size: 768
- Transformer layers: 6
- Attention heads: 12
- Activation: GELU, inherited from DistilBERT
- Saved weights: approximately 260.8 MB
- Base revision:
6ea81172465e8b0ad3fddeed32b986cdcdcffcf0
Training data
- Dataset:
Hasanblanka/polygraf-ner-stage4-final-dataset - Train records: 1,954
- Test records: 472
- Validation records: 0
- Earlier train records moved to test: 0
- Exact train/test text leakage: 0
- Global Voices article-group leakage: 0
The test split combines 127 frozen earlier test records with 345 newly held-out Global Voices records. The newly held-out records were selected only from previously unseen article groups.
Training configuration
- Epochs: 4
- Batch size: 8
- Gradient accumulation: 2
- Effective batch size: 16
- Optimizer steps: 492
- Optimizer: AdamW
- Learning rate:
5e-5 - Weight decay:
0.01 - Warmup ratio:
0.1 - Maximum gradient norm:
1.0 - Scheduler: linear schedule with warmup
- Seed: 42
- Maximum sequence length: 512
- Truncation: disabled
- Device: CPU
- Training runtime: approximately 1 hour 45 minutes
No validation split was used. The held-out test split did not contribute gradients.
Evaluation
| Metric | Train | Combined test |
|---|---|---|
| Precision | 0.9041 | 0.7077 |
| Recall | 0.9511 | 0.8100 |
| Entity F1 | 0.9270 | 0.7554 |
| Macro F1 | 0.9019 | 0.7022 |
| Token accuracy | 0.9919 | 0.9571 |
Test results by entity type:
| Label | Precision | Recall | F1 |
|---|---|---|---|
| PERSON | 0.5815 | 0.7133 | 0.6407 |
| ORGANIZATION | 0.5951 | 0.7295 | 0.6555 |
| LOCATION | 0.7308 | 0.8732 | 0.7957 |
| TIMEDATE | 0.8549 | 0.9083 | 0.8808 |
| PRODUCT | 0.6752 | 0.6529 | 0.6639 |
| WORKOFART | 0.2848 | 0.3849 | 0.3274 |
| JOB | 0.7057 | 0.8098 | 0.7542 |
| AMOUNT | 0.8900 | 0.9100 | 0.8999 |
For an apples-to-apples comparison, entity F1 on the same 127 frozen records rose from 0.5530 in Stage 4 Part 1 to 0.5967 in this final model. The newly held-out 345-record Global Voices subset scored 0.7929.
Frozen wild-sample QA
The same 10 manually annotated samples used to diagnose the Stage 3 model were evaluated again:
- Precision: 0.7872
- Recall: 0.8605
- Entity F1: 0.8222
- Macro F1: 0.7794
Stage 3 scored 0.5376 F1 on this set. The final model improved organization, job, temporal, and excessive work-title behavior, while product boundaries and some organization/location distinctions remain unresolved.
Usage
from transformers import pipeline
model_id = "Hasanblanka/polygraf-ner-stage4-final-model"
ner = pipeline(
task="token-classification",
model=model_id,
tokenizer=model_id,
aggregation_strategy="simple",
)
text = "Maya Ortiz joined Northstar Robotics in Lisbon on March 12, 2026."
for entity in ner(text):
print(entity)
Reproducibility files
stage4_part2_model.yaml: pinned training configuration.metrics/train_metrics.json: complete train metrics.metrics/test_metrics.json: complete combined-test metrics.metrics/per_label_*.json: label-level metrics.metrics/manual_test_metrics.json: frozen wild-test metrics.metrics/training_history.json: per-epoch loss and learning-rate history.metrics/training_summary.json: environment, hashes, runtime, and outputs.
Limitations
- This model is not intended for production use.
WORKOFARTremains the weakest label and shows boundary fragmentation, false positives, and missed titles.PERSONandORGANIZATIONgeneralization is weaker than the temporal and amount labels.- Train F1 is substantially higher than test F1, indicating a remaining generalization gap.
- Training data is English and news-heavy.
- Nested entities are not supported.
- The 10-record wild test is diagnostic, not statistically representative.
License and data terms
The base DistilBERT checkpoint is Apache-2.0. The fine-tuning dataset combines
evaluation-use-only starter data, CC BY / CC BY-SA Open Newswire material,
CC BY 3.0 Global Voices passages, and synthetic records. This model is
published with license: other for evaluation and research. Users must review
the full licensing and attribution notes in the Stage 4 dataset repository.