NER model for definition component recognition in German scientific texts
gbert-base-definitions_ner is a NER model (token classification) in the scientific domain in German, finetuned from the model gbert-base. It was trained using a custom annotated dataset of around 10,000 training and 2,000 test examples containing definition- and non-definition-related sentences from wikipedia articles in german.
The model is specifically designed to recognize and classify components of definitions, using the following entity labels:
- DF: Definiendum (the term being defined)
- VF: Definitor (the verb or phrase introducing the definition)
- GF: Definiens (the explanation or meaning)
Training was conducted using a standard NER objective. The model achieves an F1 score of approximately 83% on the evaluation set.
Here are the overall final metrics on the test dataset after 5 epochs of training:
- f1: 0.8314215362440694
- precision: 0.8272977172224214
- recall: 0.8355866731047803
- loss: 0.28708234429359436
Model Performance Comparision on wiki_definitions_de_multitask:
| Model | Precision | Recall | F1 Score | Eval Samples per Second | Epoch |
|---|---|---|---|---|---|
| distilbert-base-multilingual-cased-definitions_ner | 80.76 | 81.74 | 81.25 | 457.53 | 5.0 |
| scibert_scivocab_cased-definitions_ner | 80.54 | 82.11 | 81.32 | 236.61 | 4.0 |
| GottBERT_base_best-definitions_ner | 82.98 | 82.81 | 82.90 | 272.26 | 5.0 |
| xlm-roberta-base-definitions_ner | 81.90 | 83.35 | 82.62 | 241.21 | 5.0 |
| gbert-base-definitions_ner | 82.73 | 83.56 | 83.14 | 278.87 | 5.0 |
| gbert-large-definitions_ner | 80.67 | 83.36 | 81.99 | 109.83 | 2.0 |