model_v1_complete_training_wt_init_48_tiny
This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:
- Loss: 3.6497
- Accuracy: 0.3896
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 10
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10000
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 6.0224 | 0.33 | 30000 | 5.9447 | 0.1517 |
| 5.1853 | 0.66 | 60000 | 4.9635 | 0.2615 |
| 4.9483 | 0.98 | 90000 | 4.7016 | 0.2830 |
| 4.7679 | 1.31 | 120000 | 4.5154 | 0.2992 |
| 4.6448 | 1.64 | 150000 | 4.3884 | 0.3100 |
| 4.5688 | 1.97 | 180000 | 4.3095 | 0.3175 |
| 4.5102 | 2.29 | 210000 | 4.2511 | 0.3236 |
| 4.4662 | 2.62 | 240000 | 4.2038 | 0.3294 |
| 4.4269 | 2.95 | 270000 | 4.1677 | 0.3336 |
| 4.3982 | 3.28 | 300000 | 4.1367 | 0.3370 |
| 4.3714 | 3.6 | 330000 | 4.1103 | 0.3399 |
| 4.3493 | 3.93 | 360000 | 4.0869 | 0.3423 |
| 4.3303 | 4.26 | 390000 | 4.0680 | 0.3439 |
| 4.3131 | 4.59 | 420000 | 4.0467 | 0.3461 |
| 4.2875 | 4.92 | 450000 | 4.0292 | 0.3477 |
| 4.2629 | 5.24 | 480000 | 4.0109 | 0.3497 |
| 4.2413 | 5.57 | 510000 | 3.9931 | 0.3515 |
| 4.2282 | 5.9 | 540000 | 3.9759 | 0.3536 |
| 4.2003 | 6.23 | 570000 | 3.9608 | 0.3551 |
| 4.1867 | 6.55 | 600000 | 3.9445 | 0.3571 |
| 4.1607 | 6.88 | 630000 | 3.9273 | 0.3590 |
| 4.1511 | 7.21 | 660000 | 3.9130 | 0.3606 |
| 4.1335 | 7.54 | 690000 | 3.8971 | 0.3622 |
| 4.1158 | 7.87 | 720000 | 3.8798 | 0.3642 |
| 4.097 | 8.19 | 750000 | 3.8635 | 0.3663 |
| 4.0831 | 8.52 | 780000 | 3.8494 | 0.3679 |
| 4.0756 | 8.85 | 810000 | 3.8334 | 0.3696 |
| 4.0533 | 9.18 | 840000 | 3.8201 | 0.3712 |
| 4.0517 | 9.5 | 870000 | 3.8080 | 0.3724 |
| 4.0325 | 9.83 | 900000 | 3.7975 | 0.3734 |
| 4.0142 | 10.16 | 930000 | 3.7872 | 0.3748 |
| 4.0124 | 10.49 | 960000 | 3.7788 | 0.3759 |
| 4.0076 | 10.81 | 990000 | 3.7679 | 0.3767 |
| 3.9919 | 11.14 | 1020000 | 3.7609 | 0.3775 |
| 3.9888 | 11.47 | 1050000 | 3.7550 | 0.3783 |
| 3.9796 | 11.8 | 1080000 | 3.7481 | 0.3789 |
| 3.9742 | 12.13 | 1110000 | 3.7414 | 0.3796 |
| 3.9667 | 12.45 | 1140000 | 3.7370 | 0.3802 |
| 3.9652 | 12.78 | 1170000 | 3.7289 | 0.3810 |
| 3.9548 | 13.11 | 1200000 | 3.7278 | 0.3812 |
| 3.9556 | 13.44 | 1230000 | 3.7213 | 0.3817 |
| 3.9444 | 13.76 | 1260000 | 3.7152 | 0.3825 |
| 3.9428 | 14.09 | 1290000 | 3.7120 | 0.3827 |
| 3.9424 | 14.42 | 1320000 | 3.7072 | 0.3834 |
| 3.9389 | 14.75 | 1350000 | 3.7047 | 0.3836 |
| 3.936 | 15.07 | 1380000 | 3.6998 | 0.3844 |
| 3.9246 | 15.4 | 1410000 | 3.6968 | 0.3847 |
| 3.9281 | 15.73 | 1440000 | 3.6925 | 0.3851 |
| 3.9177 | 16.06 | 1470000 | 3.6916 | 0.3849 |
| 3.9216 | 16.39 | 1500000 | 3.6870 | 0.3855 |
| 3.9141 | 16.71 | 1530000 | 3.6822 | 0.3863 |
| 3.9154 | 17.04 | 1560000 | 3.6804 | 0.3864 |
| 3.9145 | 17.37 | 1590000 | 3.6795 | 0.3863 |
| 3.9103 | 17.7 | 1620000 | 3.6734 | 0.3869 |
| 3.9079 | 18.02 | 1650000 | 3.6724 | 0.3873 |
| 3.901 | 18.35 | 1680000 | 3.6707 | 0.3872 |
| 3.9015 | 18.68 | 1710000 | 3.6695 | 0.3873 |
| 3.8987 | 19.01 | 1740000 | 3.6672 | 0.3877 |
| 3.8929 | 19.33 | 1770000 | 3.6647 | 0.3878 |
| 3.892 | 19.66 | 1800000 | 3.6609 | 0.3884 |
| 3.8906 | 19.99 | 1830000 | 3.6595 | 0.3886 |
| 3.8923 | 20.32 | 1860000 | 3.6594 | 0.3885 |
| 3.8901 | 20.65 | 1890000 | 3.6541 | 0.3893 |
| 3.8853 | 20.97 | 1920000 | 3.6539 | 0.3891 |
| 3.8808 | 21.3 | 1950000 | 3.6527 | 0.3894 |
| 3.8835 | 21.63 | 1980000 | 3.6497 | 0.3896 |
Framework versions
- Transformers 4.30.2
- Pytorch 1.14.0a0+410ce96
- Datasets 2.13.0
- Tokenizers 0.13.3