eu-pii-multi-mini-preview
!IMPORTANT! This is a preview model currently under training. Take extra caution when deploying it in any production setting.
Multilingual (24 EU languages) PII token-classification model, fine-tuned from nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large (34 entity classes, BIO scheme).
Results
| split | macro P | macro R | macro F1 (w/o O) |
|---|---|---|---|
| valid | 0.9082 | 0.8970 | 0.8963 |
| test | 0.8080 | 0.7862 | 0.7863 |
Per-language test F1 (macro, without O)
| lang | docs | F1 |
|---|---|---|
| bg | 45 | 0.9055 |
| cs | 55 | 0.9183 |
| da | 63 | 0.9023 |
| de | 59 | 0.8919 |
| el | 62 | 0.9142 |
| en | 62 | 0.8935 |
| es | 57 | 0.8732 |
| et | 78 | 0.9061 |
| fi | 65 | 0.9429 |
| fr | 52 | 0.917 |
| ga | 56 | 0.8993 |
| hr | 58 | 0.9231 |
| hu | 58 | 0.9111 |
| it | 51 | 0.8951 |
| lt | 52 | 0.9242 |
| lv | 52 | 0.9282 |
| mt | 52 | 0.9182 |
| nl | 60 | 0.9049 |
| pl | 69 | 0.7252 |
| pt | 67 | 0.8945 |
| ro | 53 | 0.8969 |
| sk | 56 | 0.8791 |
| sl | 68 | 0.8885 |
| sv | 57 | 0.8676 |
| ALL | 1407 | 0.7863 |
Entity classes
ACCOUNT_IDENTIFIER, AUTH_SECRET, BANK_ACCOUNT_IDENTIFIER, BIOMETRIC_DATA, CONTACT_HANDLE, CRIMINAL_OFFENCE_DATA, DATE, DATE_OF_BIRTH, DEVICE_IDENTIFIER, DOCUMENT_IDENTIFIER, EMAIL_ADDRESS, ETHNIC_ORIGIN, FINANCIAL_AMOUNT, GEO_LOCATION, HEALTH_DATA, IDENTIFYING_LINK, IP_ADDRESS, LOCATION, ORGANIZATION_IDENTIFIER, ORGANIZATION_NAME, PAYMENT_CARD, PAYMENT_CARD_SECURITY, PERSON_ATTRIBUTE, PERSON_IDENTIFIER, PERSON_NAME, PERSON_ROLE_OR_TITLE, PHONE_NUMBER, POLITICAL_OPINION, POSTAL_ADDRESS, PROPER_NAME, RELIGION_OR_BELIEF, SEXUAL_ORIENTATION, TRADE_UNION_MEMBERSHIP, VEHICLE_IDENTIFIER
Training
- learning rate 3e-05, batch size 32, 5 epochs, weight decay 0.01
- early stopping on validation macro-F1 (without O)
ONNX
The onnx/ folder contains model.onnx and a dynamically int8-quantized model_quantized.onnx for CPU inference:
from optimum.onnxruntime import ORTModelForTokenClassification
model = ORTModelForTokenClassification.from_pretrained(
"bardsai/eu-pii-multi-mini-preview", subfolder="onnx", file_name="model_quantized.onnx"
)