layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6932
- Answer: {'precision': 0.7073434125269978, 'recall': 0.8096415327564895, 'f1': 0.755043227665706, 'number': 809}
- Header: {'precision': 0.25925925925925924, 'recall': 0.23529411764705882, 'f1': 0.24669603524229072, 'number': 119}
- Question: {'precision': 0.7767158992180713, 'recall': 0.8394366197183099, 'f1': 0.8068592057761733, 'number': 1065}
- Overall Precision: 0.7217
- Overall Recall: 0.7913
- Overall F1: 0.7549
- Overall Accuracy: 0.8047
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.8958 | 1.0 | 5 | 1.7117 | {'precision': 0.015232292460015232, 'recall': 0.024721878862793572, 'f1': 0.01885014137606032, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.11567164179104478, 'recall': 0.11643192488262911, 'f1': 0.11605053813757606, 'number': 1065} | 0.0601 | 0.0723 | 0.0656 | 0.3457 |
| 1.632 | 2.0 | 10 | 1.4951 | {'precision': 0.027700831024930747, 'recall': 0.037082818294190356, 'f1': 0.03171247357293869, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3054968287526427, 'recall': 0.27136150234741785, 'f1': 0.2874191944306315, 'number': 1065} | 0.1571 | 0.1601 | 0.1586 | 0.4391 |
| 1.4434 | 3.0 | 15 | 1.2975 | {'precision': 0.1747173689619733, 'recall': 0.21013597033374537, 'f1': 0.19079685746352415, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.39224704336399474, 'recall': 0.5605633802816902, 'f1': 0.46153846153846156, 'number': 1065} | 0.3074 | 0.3848 | 0.3418 | 0.5806 |
| 1.2417 | 4.0 | 20 | 1.1193 | {'precision': 0.3728470111448835, 'recall': 0.45488257107540175, 'f1': 0.40979955456570155, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5278207109737248, 'recall': 0.6413145539906103, 'f1': 0.5790589232725731, 'number': 1065} | 0.4608 | 0.5273 | 0.4918 | 0.6383 |
| 1.0407 | 5.0 | 25 | 0.9770 | {'precision': 0.48043478260869565, 'recall': 0.546353522867738, 'f1': 0.5112781954887218, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5961698584512906, 'recall': 0.672300469483568, 'f1': 0.6319505736981466, 'number': 1065} | 0.5447 | 0.5810 | 0.5623 | 0.6890 |
| 0.8977 | 6.0 | 30 | 0.8662 | {'precision': 0.559375, 'recall': 0.6637824474660075, 'f1': 0.6071226681741096, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6157517899761337, 'recall': 0.7267605633802817, 'f1': 0.6666666666666667, 'number': 1065} | 0.5845 | 0.6578 | 0.6190 | 0.7313 |
| 0.787 | 7.0 | 35 | 0.8246 | {'precision': 0.6031589338598223, 'recall': 0.7552533992583437, 'f1': 0.6706915477497257, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6820194862710364, 'recall': 0.7230046948356808, 'f1': 0.7019143117593437, 'number': 1065} | 0.6332 | 0.6929 | 0.6617 | 0.7434 |
| 0.6913 | 8.0 | 40 | 0.7630 | {'precision': 0.6112820512820513, 'recall': 0.7367119901112484, 'f1': 0.6681614349775784, 'number': 809} | {'precision': 0.02702702702702703, 'recall': 0.01680672268907563, 'f1': 0.02072538860103627, 'number': 119} | {'precision': 0.6604795050270689, 'recall': 0.8018779342723005, 'f1': 0.7243426632739611, 'number': 1065} | 0.6200 | 0.7285 | 0.6699 | 0.7753 |
| 0.6196 | 9.0 | 45 | 0.7427 | {'precision': 0.629145728643216, 'recall': 0.7737948084054388, 'f1': 0.6940133037694012, 'number': 809} | {'precision': 0.08108108108108109, 'recall': 0.05042016806722689, 'f1': 0.062176165803108814, 'number': 119} | {'precision': 0.7155247181266262, 'recall': 0.7746478873239436, 'f1': 0.7439134355275022, 'number': 1065} | 0.6557 | 0.7311 | 0.6913 | 0.7750 |
| 0.57 | 10.0 | 50 | 0.7250 | {'precision': 0.6613588110403397, 'recall': 0.7700865265760197, 'f1': 0.7115933752141633, 'number': 809} | {'precision': 0.14285714285714285, 'recall': 0.09243697478991597, 'f1': 0.11224489795918367, 'number': 119} | {'precision': 0.6997538966365874, 'recall': 0.8009389671361502, 'f1': 0.7469352014010507, 'number': 1065} | 0.6644 | 0.7461 | 0.7029 | 0.7852 |
| 0.5314 | 11.0 | 55 | 0.7002 | {'precision': 0.6476578411405295, 'recall': 0.7861557478368356, 'f1': 0.7102177554438861, 'number': 809} | {'precision': 0.16216216216216217, 'recall': 0.10084033613445378, 'f1': 0.12435233160621763, 'number': 119} | {'precision': 0.7123745819397993, 'recall': 0.8, 'f1': 0.7536488279522335, 'number': 1065} | 0.6661 | 0.7526 | 0.7067 | 0.7901 |
| 0.4856 | 12.0 | 60 | 0.6901 | {'precision': 0.682454251883746, 'recall': 0.7836835599505563, 'f1': 0.7295742232451092, 'number': 809} | {'precision': 0.2717391304347826, 'recall': 0.21008403361344538, 'f1': 0.23696682464454974, 'number': 119} | {'precision': 0.7309602649006622, 'recall': 0.8291079812206573, 'f1': 0.7769467663880335, 'number': 1065} | 0.6918 | 0.7737 | 0.7305 | 0.8001 |
| 0.4486 | 13.0 | 65 | 0.6915 | {'precision': 0.6917211328976035, 'recall': 0.7849196538936959, 'f1': 0.7353792704111175, 'number': 809} | {'precision': 0.25806451612903225, 'recall': 0.20168067226890757, 'f1': 0.22641509433962265, 'number': 119} | {'precision': 0.7395659432387313, 'recall': 0.831924882629108, 'f1': 0.7830313742819267, 'number': 1065} | 0.6994 | 0.7752 | 0.7354 | 0.8011 |
| 0.4177 | 14.0 | 70 | 0.6835 | {'precision': 0.6862326574172892, 'recall': 0.7948084054388134, 'f1': 0.736540664375716, 'number': 809} | {'precision': 0.24731182795698925, 'recall': 0.19327731092436976, 'f1': 0.2169811320754717, 'number': 119} | {'precision': 0.7416666666666667, 'recall': 0.8356807511737089, 'f1': 0.7858719646799116, 'number': 1065} | 0.6978 | 0.7807 | 0.7369 | 0.8026 |
| 0.3927 | 15.0 | 75 | 0.6891 | {'precision': 0.7016393442622951, 'recall': 0.7935723114956736, 'f1': 0.7447795823665894, 'number': 809} | {'precision': 0.2358490566037736, 'recall': 0.21008403361344538, 'f1': 0.22222222222222224, 'number': 119} | {'precision': 0.7632478632478632, 'recall': 0.8384976525821596, 'f1': 0.7991051454138701, 'number': 1065} | 0.7120 | 0.7827 | 0.7457 | 0.8038 |
| 0.3743 | 16.0 | 80 | 0.6904 | {'precision': 0.7064622124863089, 'recall': 0.7972805933250927, 'f1': 0.7491289198606274, 'number': 809} | {'precision': 0.27450980392156865, 'recall': 0.23529411764705882, 'f1': 0.2533936651583711, 'number': 119} | {'precision': 0.7675814751286449, 'recall': 0.8403755868544601, 'f1': 0.8023307933662035, 'number': 1065} | 0.7189 | 0.7868 | 0.7513 | 0.8034 |
| 0.3683 | 17.0 | 85 | 0.6928 | {'precision': 0.7037837837837838, 'recall': 0.8046971569839307, 'f1': 0.7508650519031143, 'number': 809} | {'precision': 0.2545454545454545, 'recall': 0.23529411764705882, 'f1': 0.2445414847161572, 'number': 119} | {'precision': 0.7735191637630662, 'recall': 0.8338028169014085, 'f1': 0.8025305015815634, 'number': 1065} | 0.7178 | 0.7863 | 0.7505 | 0.8035 |
| 0.3553 | 18.0 | 90 | 0.6940 | {'precision': 0.7045454545454546, 'recall': 0.8046971569839307, 'f1': 0.7512983266012695, 'number': 809} | {'precision': 0.25925925925925924, 'recall': 0.23529411764705882, 'f1': 0.24669603524229072, 'number': 119} | {'precision': 0.7739130434782608, 'recall': 0.8356807511737089, 'f1': 0.8036117381489841, 'number': 1065} | 0.7191 | 0.7873 | 0.7516 | 0.8035 |
| 0.3471 | 19.0 | 95 | 0.6932 | {'precision': 0.7071583514099783, 'recall': 0.8059332509270705, 'f1': 0.753321779318313, 'number': 809} | {'precision': 0.2545454545454545, 'recall': 0.23529411764705882, 'f1': 0.2445414847161572, 'number': 119} | {'precision': 0.7746967071057193, 'recall': 0.8394366197183099, 'f1': 0.8057683641279857, 'number': 1065} | 0.7200 | 0.7898 | 0.7533 | 0.8043 |
| 0.3386 | 20.0 | 100 | 0.6932 | {'precision': 0.7073434125269978, 'recall': 0.8096415327564895, 'f1': 0.755043227665706, 'number': 809} | {'precision': 0.25925925925925924, 'recall': 0.23529411764705882, 'f1': 0.24669603524229072, 'number': 119} | {'precision': 0.7767158992180713, 'recall': 0.8394366197183099, 'f1': 0.8068592057761733, 'number': 1065} | 0.7217 | 0.7913 | 0.7549 | 0.8047 |
Framework versions
- Transformers 4.45.1
- Pytorch 2.4.0
- Datasets 3.0.1
- Tokenizers 0.20.0