8_bar_lmd_clean_custom_final_token_scheme_epochs10
This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 5.0474
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.005
- train_batch_size: 48
- eval_batch_size: 32
- seed: 1
- gradient_accumulation_steps: 2
- total_train_batch_size: 96
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.01
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 8.2004 | 0.03 | 10 | 5.8581 |
| 5.8284 | 0.06 | 20 | 5.7712 |
| 5.7438 | 0.09 | 30 | 5.7220 |
| 5.682 | 0.11 | 40 | 5.6185 |
| 5.6048 | 0.14 | 50 | 5.5303 |
| 5.5522 | 0.17 | 60 | 5.5000 |
| 5.5071 | 0.2 | 70 | 5.4540 |
| 5.4512 | 0.23 | 80 | 5.3570 |
| 5.2932 | 0.26 | 90 | 5.1704 |
| 5.1931 | 0.29 | 100 | 5.1101 |
| 5.109 | 0.32 | 110 | 5.0844 |
| 5.1019 | 0.34 | 120 | 5.0580 |
| 5.0702 | 0.37 | 130 | 5.0452 |
| 5.0727 | 0.4 | 140 | 5.0375 |
| 5.0654 | 0.43 | 150 | 5.0376 |
| 5.0625 | 0.46 | 160 | 5.0292 |
| 5.0426 | 0.49 | 170 | 5.0376 |
| 5.0428 | 0.52 | 180 | 5.0328 |
| 5.0153 | 0.54 | 190 | 5.0280 |
| 5.0307 | 0.57 | 200 | 5.0256 |
| 5.0321 | 0.6 | 210 | 5.0202 |
| 5.0638 | 0.63 | 220 | 5.0150 |
| 5.0363 | 0.66 | 230 | 5.0077 |
| 5.0151 | 0.69 | 240 | 5.0067 |
| 5.0359 | 0.72 | 250 | 5.0131 |
| 4.9987 | 0.74 | 260 | 4.9949 |
| 5.0128 | 0.77 | 270 | 4.9826 |
| 5.0164 | 0.8 | 280 | 4.9872 |
| 4.9941 | 0.83 | 290 | 4.9869 |
| 5.0258 | 0.86 | 300 | 4.9968 |
| 5.0132 | 0.89 | 310 | 5.0012 |
| 5.0568 | 0.92 | 320 | 5.0877 |
| 5.1161 | 0.95 | 330 | 5.0727 |
| 5.1321 | 0.97 | 340 | 5.1333 |
| 5.1533 | 1.0 | 350 | 5.1079 |
| 5.1469 | 1.03 | 360 | 5.0910 |
| 5.1172 | 1.06 | 370 | 5.0900 |
| 5.1259 | 1.09 | 380 | 5.1383 |
| 5.1742 | 1.12 | 390 | 5.1814 |
| 5.2056 | 1.15 | 400 | 5.1766 |
| 5.1964 | 1.17 | 410 | 5.1935 |
| 5.2422 | 1.2 | 420 | 5.2032 |
| 5.2544 | 1.23 | 430 | 5.1723 |
| 5.2853 | 1.26 | 440 | 5.2317 |
| 5.302 | 1.29 | 450 | 5.3312 |
| 5.319 | 1.32 | 460 | 5.2069 |
| 5.2633 | 1.35 | 470 | 5.2114 |
| 5.2548 | 1.38 | 480 | 5.2350 |
| 5.294 | 1.4 | 490 | 5.3470 |
| 5.2876 | 1.43 | 500 | 5.1773 |
| 5.2857 | 1.46 | 510 | 5.2445 |
| 5.3095 | 1.49 | 520 | 5.2099 |
| 5.2322 | 1.52 | 530 | 5.2158 |
| 5.215 | 1.55 | 540 | 5.1505 |
| 5.2248 | 1.58 | 550 | 5.1520 |
| 5.2123 | 1.6 | 560 | 5.1412 |
| 5.2098 | 1.63 | 570 | 5.1431 |
| 5.2088 | 1.66 | 580 | 5.1443 |
| 5.2007 | 1.69 | 590 | 5.1595 |
| 5.212 | 1.72 | 600 | 5.2016 |
| 5.2143 | 1.75 | 610 | 5.1499 |
| 5.2152 | 1.78 | 620 | 5.1333 |
| 5.2003 | 1.81 | 630 | 5.1810 |
| 5.2761 | 1.83 | 640 | 5.1993 |
| 5.2707 | 1.86 | 650 | 5.1884 |
| 5.2622 | 1.89 | 660 | 5.1815 |
| 5.242 | 1.92 | 670 | 5.1830 |
| 5.2705 | 1.95 | 680 | 5.2060 |
| 5.28 | 1.98 | 690 | 5.1905 |
| 5.2443 | 2.01 | 700 | 5.1681 |
| 5.2252 | 2.03 | 710 | 5.1609 |
| 5.2256 | 2.06 | 720 | 5.1565 |
| 5.2043 | 2.09 | 730 | 5.1541 |
| 5.2366 | 2.12 | 740 | 5.1640 |
| 5.2441 | 2.15 | 750 | 5.1742 |
| 5.2585 | 2.18 | 760 | 5.1817 |
| 5.2182 | 2.21 | 770 | 5.1588 |
| 5.2153 | 2.23 | 780 | 5.1710 |
| 5.218 | 2.26 | 790 | 5.1414 |
| 5.2067 | 2.29 | 800 | 5.1374 |
| 5.1897 | 2.32 | 810 | 5.1294 |
| 5.195 | 2.35 | 820 | 5.1315 |
| 5.2114 | 2.38 | 830 | 5.1335 |
| 5.2061 | 2.41 | 840 | 5.1366 |
| 5.1933 | 2.44 | 850 | 5.1307 |
| 5.1856 | 2.46 | 860 | 5.1311 |
| 5.1972 | 2.49 | 870 | 5.1317 |
| 5.1958 | 2.52 | 880 | 5.1356 |
| 5.2131 | 2.55 | 890 | 5.1300 |
| 5.1906 | 2.58 | 900 | 5.1177 |
| 5.1984 | 2.61 | 910 | 5.1166 |
| 5.1799 | 2.64 | 920 | 5.1297 |
| 5.1874 | 2.66 | 930 | 5.1328 |
| 5.1896 | 2.69 | 940 | 5.1219 |
| 5.1864 | 2.72 | 950 | 5.1286 |
| 5.203 | 2.75 | 960 | 5.1238 |
| 5.2038 | 2.78 | 970 | 5.1183 |
| 5.1992 | 2.81 | 980 | 5.1116 |
| 5.1733 | 2.84 | 990 | 5.1099 |
| 5.1604 | 2.87 | 1000 | 5.1101 |
| 5.1657 | 2.89 | 1010 | 5.1067 |
| 5.1594 | 2.92 | 1020 | 5.0977 |
| 5.1501 | 2.95 | 1030 | 5.0945 |
| 5.1485 | 2.98 | 1040 | 5.0897 |
| 5.164 | 3.01 | 1050 | 5.0835 |
| 5.1394 | 3.04 | 1060 | 5.0955 |
| 5.1472 | 3.07 | 1070 | 5.1000 |
| 5.2015 | 3.09 | 1080 | 5.0947 |
| 5.1665 | 3.12 | 1090 | 5.0916 |
| 5.1483 | 3.15 | 1100 | 5.0958 |
| 5.1753 | 3.18 | 1110 | 5.0902 |
| 5.1565 | 3.21 | 1120 | 5.0886 |
| 5.1572 | 3.24 | 1130 | 5.0876 |
| 5.1193 | 3.27 | 1140 | 5.0900 |
| 5.144 | 3.3 | 1150 | 5.0807 |
| 5.1362 | 3.32 | 1160 | 5.0913 |
| 5.1574 | 3.35 | 1170 | 5.0780 |
| 5.1381 | 3.38 | 1180 | 5.0738 |
| 5.1405 | 3.41 | 1190 | 5.0739 |
| 5.1463 | 3.44 | 1200 | 5.0739 |
| 5.1324 | 3.47 | 1210 | 5.0729 |
| 5.1102 | 3.5 | 1220 | 5.0703 |
| 5.1575 | 3.52 | 1230 | 5.0700 |
| 5.125 | 3.55 | 1240 | 5.0674 |
| 5.1391 | 3.58 | 1250 | 5.0673 |
| 5.1405 | 3.61 | 1260 | 5.0678 |
| 5.141 | 3.64 | 1270 | 5.0708 |
| 5.1059 | 3.67 | 1280 | 5.0719 |
| 5.1423 | 3.7 | 1290 | 5.0719 |
| 5.1098 | 3.72 | 1300 | 5.0698 |
| 5.1165 | 3.75 | 1310 | 5.0674 |
| 5.1249 | 3.78 | 1320 | 5.0660 |
| 5.1129 | 3.81 | 1330 | 5.0675 |
| 5.1469 | 3.84 | 1340 | 5.0677 |
| 5.1215 | 3.87 | 1350 | 5.0688 |
| 5.1392 | 3.9 | 1360 | 5.0685 |
| 5.1355 | 3.93 | 1370 | 5.0674 |
| 5.1372 | 3.95 | 1380 | 5.0666 |
| 5.1237 | 3.98 | 1390 | 5.0641 |
| 5.1452 | 4.01 | 1400 | 5.0623 |
| 5.117 | 4.04 | 1410 | 5.0642 |
| 5.1467 | 4.07 | 1420 | 5.0604 |
| 5.1221 | 4.1 | 1430 | 5.0590 |
| 5.0959 | 4.13 | 1440 | 5.0540 |
| 5.1088 | 4.15 | 1450 | 5.0538 |
| 5.116 | 4.18 | 1460 | 5.0542 |
| 5.1293 | 4.21 | 1470 | 5.0542 |
| 5.1337 | 4.24 | 1480 | 5.0526 |
| 5.1154 | 4.27 | 1490 | 5.0522 |
| 5.1196 | 4.3 | 1500 | 5.0538 |
| 5.1122 | 4.33 | 1510 | 5.0515 |
| 5.094 | 4.36 | 1520 | 5.0503 |
| 5.116 | 4.38 | 1530 | 5.0505 |
| 5.1142 | 4.41 | 1540 | 5.0520 |
| 5.1106 | 4.44 | 1550 | 5.0517 |
| 5.1023 | 4.47 | 1560 | 5.0502 |
| 5.1153 | 4.5 | 1570 | 5.0497 |
| 5.1096 | 4.53 | 1580 | 5.0493 |
| 5.1331 | 4.56 | 1590 | 5.0503 |
| 5.1178 | 4.58 | 1600 | 5.0506 |
| 5.0984 | 4.61 | 1610 | 5.0503 |
| 5.0992 | 4.64 | 1620 | 5.0493 |
| 5.0888 | 4.67 | 1630 | 5.0487 |
| 5.1153 | 4.7 | 1640 | 5.0484 |
| 5.1102 | 4.73 | 1650 | 5.0480 |
| 5.1143 | 4.76 | 1660 | 5.0478 |
| 5.1012 | 4.79 | 1670 | 5.0477 |
| 5.104 | 4.81 | 1680 | 5.0478 |
| 5.1129 | 4.84 | 1690 | 5.0475 |
| 5.0908 | 4.87 | 1700 | 5.0475 |
| 5.091 | 4.9 | 1710 | 5.0474 |
| 5.1115 | 4.93 | 1720 | 5.0474 |
| 5.1267 | 4.96 | 1730 | 5.0474 |
| 5.1245 | 4.99 | 1740 | 5.0474 |
Framework versions
- Transformers 4.36.0
- Pytorch 2.1.0
- Datasets 2.15.0
- Tokenizers 0.15.1