trackmania-tmnf-gpt2-safefilter3
This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.1672
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3.0
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.6212 | 0.0132 | 200 | 1.6541 |
| 1.6141 | 0.0264 | 400 | 1.5709 |
| 1.5451 | 0.0396 | 600 | 1.5353 |
| 1.4576 | 0.0528 | 800 | 1.4997 |
| 1.4424 | 0.0660 | 1000 | 1.4647 |
| 1.4506 | 0.0792 | 1200 | 1.4334 |
| 1.4144 | 0.0924 | 1400 | 1.4217 |
| 1.4381 | 0.1056 | 1600 | 1.4010 |
| 1.4226 | 0.1188 | 1800 | 1.3875 |
| 1.3720 | 0.1320 | 2000 | 1.3717 |
| 1.3816 | 0.1452 | 2200 | 1.3621 |
| 1.3280 | 0.1584 | 2400 | 1.3547 |
| 1.3089 | 0.1716 | 2600 | 1.3439 |
| 1.2891 | 0.1848 | 2800 | 1.3376 |
| 1.3154 | 0.1980 | 3000 | 1.3298 |
| 1.3695 | 0.2112 | 3200 | 1.3206 |
| 1.3573 | 0.2244 | 3400 | 1.3207 |
| 1.3373 | 0.2376 | 3600 | 1.3132 |
| 1.3252 | 0.2508 | 3800 | 1.3080 |
| 1.3074 | 0.2640 | 4000 | 1.3028 |
| 1.2535 | 0.2772 | 4200 | 1.2995 |
| 1.2647 | 0.2904 | 4400 | 1.2942 |
| 1.3423 | 0.3036 | 4600 | 1.2901 |
| 1.3118 | 0.3168 | 4800 | 1.2894 |
| 1.3550 | 0.3300 | 5000 | 1.2863 |
| 1.2639 | 0.3432 | 5200 | 1.2821 |
| 1.2669 | 0.3564 | 5400 | 1.2796 |
| 1.2767 | 0.3696 | 5600 | 1.2745 |
| 1.2482 | 0.3828 | 5800 | 1.2718 |
| 1.2791 | 0.3960 | 6000 | 1.2725 |
| 1.2611 | 0.4092 | 6200 | 1.2690 |
| 1.2944 | 0.4224 | 6400 | 1.2658 |
| 1.2797 | 0.4356 | 6600 | 1.2682 |
| 1.2865 | 0.4488 | 6800 | 1.2603 |
| 1.2478 | 0.4620 | 7000 | 1.2589 |
| 1.3215 | 0.4752 | 7200 | 1.2584 |
| 1.2904 | 0.4884 | 7400 | 1.2570 |
| 1.2425 | 0.5016 | 7600 | 1.2536 |
| 1.1775 | 0.5148 | 7800 | 1.2511 |
| 1.2073 | 0.5280 | 8000 | 1.2501 |
| 1.2868 | 0.5412 | 8200 | 1.2486 |
| 1.2881 | 0.5544 | 8400 | 1.2483 |
| 1.2307 | 0.5676 | 8600 | 1.2463 |
| 1.2326 | 0.5808 | 8800 | 1.2450 |
| 1.2422 | 0.5940 | 9000 | 1.2414 |
| 1.2444 | 0.6072 | 9200 | 1.2435 |
| 1.2426 | 0.6204 | 9400 | 1.2389 |
| 1.2277 | 0.6336 | 9600 | 1.2386 |
| 1.2287 | 0.6468 | 9800 | 1.2395 |
| 1.2108 | 0.6600 | 10000 | 1.2367 |
| 1.2359 | 0.6732 | 10200 | 1.2357 |
| 1.1887 | 0.6864 | 10400 | 1.2341 |
| 1.2531 | 0.6996 | 10600 | 1.2334 |
| 1.2549 | 0.7128 | 10800 | 1.2312 |
| 1.2419 | 0.7260 | 11000 | 1.2305 |
| 1.2123 | 0.7392 | 11200 | 1.2305 |
| 1.2113 | 0.7524 | 11400 | 1.2293 |
| 1.2070 | 0.7656 | 11600 | 1.2292 |
| 1.2724 | 0.7788 | 11800 | 1.2262 |
| 1.1557 | 0.7920 | 12000 | 1.2269 |
| 1.2546 | 0.8052 | 12200 | 1.2232 |
| 1.2196 | 0.8184 | 12400 | 1.2246 |
| 1.2423 | 0.8316 | 12600 | 1.2242 |
| 1.2090 | 0.8448 | 12800 | 1.2248 |
| 1.2427 | 0.8580 | 13000 | 1.2226 |
| 1.2327 | 0.8712 | 13200 | 1.2196 |
| 1.2742 | 0.8844 | 13400 | 1.2199 |
| 1.1730 | 0.8976 | 13600 | 1.2197 |
| 1.2025 | 0.9108 | 13800 | 1.2182 |
| 1.2321 | 0.9240 | 14000 | 1.2186 |
| 1.2462 | 0.9372 | 14200 | 1.2175 |
| 1.2633 | 0.9504 | 14400 | 1.2152 |
| 1.2477 | 0.9636 | 14600 | 1.2143 |
| 1.2435 | 0.9768 | 14800 | 1.2145 |
| 1.2156 | 0.9900 | 15000 | 1.2140 |
| 1.1891 | 1.0032 | 15200 | 1.2140 |
| 1.1772 | 1.0164 | 15400 | 1.2137 |
| 1.2563 | 1.0296 | 15600 | 1.2115 |
| 1.2588 | 1.0428 | 15800 | 1.2119 |
| 1.1924 | 1.0560 | 16000 | 1.2112 |
| 1.2095 | 1.0692 | 16200 | 1.2083 |
| 1.1767 | 1.0824 | 16400 | 1.2089 |
| 1.2310 | 1.0956 | 16600 | 1.2089 |
| 1.1701 | 1.1088 | 16800 | 1.2076 |
| 1.2179 | 1.1220 | 17000 | 1.2074 |
| 1.1988 | 1.1352 | 17200 | 1.2082 |
| 1.2498 | 1.1484 | 17400 | 1.2048 |
| 1.2361 | 1.1616 | 17600 | 1.2056 |
| 1.2017 | 1.1748 | 17800 | 1.2048 |
| 1.2223 | 1.1880 | 18000 | 1.2039 |
| 1.2216 | 1.2012 | 18200 | 1.2028 |
| 1.2029 | 1.2144 | 18400 | 1.2006 |
| 1.2417 | 1.2276 | 18600 | 1.2029 |
| 1.1997 | 1.2408 | 18800 | 1.2014 |
| 1.1755 | 1.2540 | 19000 | 1.2007 |
| 1.1523 | 1.2672 | 19200 | 1.2005 |
| 1.2059 | 1.2804 | 19400 | 1.2007 |
| 1.2175 | 1.2936 | 19600 | 1.1989 |
| 1.1730 | 1.3068 | 19800 | 1.1992 |
| 1.2275 | 1.3200 | 20000 | 1.1967 |
| 1.1847 | 1.3332 | 20200 | 1.1980 |
| 1.2435 | 1.3464 | 20400 | 1.1977 |
| 1.1716 | 1.3596 | 20600 | 1.1961 |
| 1.2264 | 1.3728 | 20800 | 1.1964 |
| 1.2300 | 1.3860 | 21000 | 1.1976 |
| 1.1502 | 1.3992 | 21200 | 1.1957 |
| 1.1515 | 1.4124 | 21400 | 1.1953 |
| 1.1437 | 1.4256 | 21600 | 1.1955 |
| 1.1754 | 1.4388 | 21800 | 1.1927 |
| 1.2154 | 1.4520 | 22000 | 1.1937 |
| 1.2683 | 1.4652 | 22200 | 1.1926 |
| 1.1785 | 1.4784 | 22400 | 1.1936 |
| 1.1717 | 1.4916 | 22600 | 1.1934 |
| 1.2055 | 1.5048 | 22800 | 1.1931 |
| 1.1811 | 1.5180 | 23000 | 1.1916 |
| 1.1559 | 1.5312 | 23200 | 1.1916 |
| 1.1865 | 1.5444 | 23400 | 1.1902 |
| 1.2192 | 1.5576 | 23600 | 1.1904 |
| 1.1907 | 1.5707 | 23800 | 1.1888 |
| 1.1650 | 1.5839 | 24000 | 1.1892 |
| 1.1954 | 1.5971 | 24200 | 1.1894 |
| 1.1855 | 1.6103 | 24400 | 1.1903 |
| 1.2270 | 1.6235 | 24600 | 1.1890 |
| 1.1592 | 1.6367 | 24800 | 1.1879 |
| 1.2147 | 1.6499 | 25000 | 1.1876 |
| 1.1603 | 1.6631 | 25200 | 1.1879 |
| 1.2099 | 1.6763 | 25400 | 1.1860 |
| 1.1645 | 1.6895 | 25600 | 1.1863 |
| 1.1843 | 1.7027 | 25800 | 1.1867 |
| 1.1741 | 1.7159 | 26000 | 1.1861 |
| 1.1946 | 1.7291 | 26200 | 1.1856 |
| 1.1819 | 1.7423 | 26400 | 1.1836 |
| 1.1866 | 1.7555 | 26600 | 1.1834 |
| 1.2266 | 1.7687 | 26800 | 1.1840 |
| 1.2218 | 1.7819 | 27000 | 1.1838 |
| 1.2183 | 1.7951 | 27200 | 1.1831 |
| 1.1789 | 1.8083 | 27400 | 1.1826 |
| 1.2127 | 1.8215 | 27600 | 1.1820 |
| 1.2057 | 1.8347 | 27800 | 1.1815 |
| 1.2244 | 1.8479 | 28000 | 1.1825 |
| 1.2153 | 1.8611 | 28200 | 1.1813 |
| 1.2190 | 1.8743 | 28400 | 1.1800 |
| 1.2297 | 1.8875 | 28600 | 1.1822 |
| 1.1925 | 1.9007 | 28800 | 1.1809 |
| 1.1767 | 1.9139 | 29000 | 1.1795 |
| 1.2521 | 1.9271 | 29200 | 1.1801 |
| 1.1644 | 1.9403 | 29400 | 1.1810 |
| 1.1620 | 1.9535 | 29600 | 1.1795 |
| 1.2195 | 1.9667 | 29800 | 1.1796 |
| 1.1435 | 1.9799 | 30000 | 1.1788 |
| 1.1491 | 1.9931 | 30200 | 1.1790 |
| 1.1841 | 2.0063 | 30400 | 1.1787 |
| 1.2032 | 2.0195 | 30600 | 1.1785 |
| 1.1602 | 2.0327 | 30800 | 1.1775 |
| 1.1999 | 2.0459 | 31000 | 1.1779 |
| 1.2330 | 2.0591 | 31200 | 1.1771 |
| 1.2007 | 2.0723 | 31400 | 1.1769 |
| 1.1654 | 2.0855 | 31600 | 1.1771 |
| 1.2126 | 2.0987 | 31800 | 1.1771 |
| 1.1159 | 2.1119 | 32000 | 1.1767 |
| 1.1738 | 2.1251 | 32200 | 1.1765 |
| 1.2303 | 2.1383 | 32400 | 1.1766 |
| 1.1351 | 2.1515 | 32600 | 1.1766 |
| 1.2319 | 2.1647 | 32800 | 1.1761 |
| 1.1640 | 2.1779 | 33000 | 1.1753 |
| 1.1444 | 2.1911 | 33200 | 1.1753 |
| 1.1654 | 2.2043 | 33400 | 1.1750 |
| 1.1196 | 2.2175 | 33600 | 1.1754 |
| 1.1560 | 2.2307 | 33800 | 1.1741 |
| 1.1951 | 2.2439 | 34000 | 1.1746 |
| 1.2292 | 2.2571 | 34200 | 1.1746 |
| 1.1446 | 2.2703 | 34400 | 1.1753 |
| 1.2139 | 2.2835 | 34600 | 1.1733 |
| 1.1965 | 2.2967 | 34800 | 1.1734 |
| 1.1076 | 2.3099 | 35000 | 1.1737 |
| 1.1617 | 2.3231 | 35200 | 1.1730 |
| 1.1955 | 2.3363 | 35400 | 1.1720 |
| 1.1741 | 2.3495 | 35600 | 1.1726 |
| 1.1402 | 2.3627 | 35800 | 1.1718 |
| 1.1891 | 2.3759 | 36000 | 1.1724 |
| 1.1513 | 2.3891 | 36200 | 1.1730 |
| 1.1742 | 2.4023 | 36400 | 1.1719 |
| 1.1750 | 2.4155 | 36600 | 1.1715 |
| 1.2108 | 2.4287 | 36800 | 1.1718 |
| 1.1310 | 2.4419 | 37000 | 1.1714 |
| 1.1422 | 2.4551 | 37200 | 1.1714 |
| 1.2209 | 2.4683 | 37400 | 1.1705 |
| 1.1538 | 2.4815 | 37600 | 1.1711 |
| 1.1508 | 2.4947 | 37800 | 1.1716 |
| 1.2220 | 2.5079 | 38000 | 1.1706 |
| 1.1614 | 2.5211 | 38200 | 1.1697 |
| 1.1349 | 2.5343 | 38400 | 1.1713 |
| 1.0819 | 2.5475 | 38600 | 1.1717 |
| 1.1667 | 2.5607 | 38800 | 1.1704 |
| 1.1463 | 2.5739 | 39000 | 1.1700 |
| 1.1646 | 2.5871 | 39200 | 1.1701 |
| 1.2061 | 2.6003 | 39400 | 1.1699 |
| 1.1266 | 2.6135 | 39600 | 1.1697 |
| 1.2128 | 2.6267 | 39800 | 1.1695 |
| 1.1321 | 2.6399 | 40000 | 1.1696 |
| 1.1423 | 2.6531 | 40200 | 1.1695 |
| 1.1476 | 2.6663 | 40400 | 1.1685 |
| 1.1003 | 2.6795 | 40600 | 1.1699 |
| 1.1415 | 2.6927 | 40800 | 1.1688 |
| 1.1497 | 2.7059 | 41000 | 1.1692 |
| 1.1486 | 2.7191 | 41200 | 1.1683 |
| 1.2073 | 2.7323 | 41400 | 1.1684 |
| 1.1718 | 2.7455 | 41600 | 1.1683 |
| 1.1448 | 2.7587 | 41800 | 1.1686 |
| 1.1017 | 2.7719 | 42000 | 1.1685 |
| 1.1674 | 2.7851 | 42200 | 1.1687 |
| 1.1708 | 2.7983 | 42400 | 1.1679 |
| 1.1318 | 2.8115 | 42600 | 1.1682 |
| 1.2200 | 2.8247 | 42800 | 1.1681 |
| 1.1178 | 2.8379 | 43000 | 1.1677 |
| 1.1369 | 2.8511 | 43200 | 1.1680 |
| 1.1429 | 2.8643 | 43400 | 1.1680 |
| 1.1325 | 2.8775 | 43600 | 1.1675 |
| 1.2113 | 2.8907 | 43800 | 1.1672 |
| 1.1839 | 2.9039 | 44000 | 1.1675 |
| 1.1246 | 2.9171 | 44200 | 1.1676 |
| 1.1548 | 2.9303 | 44400 | 1.1673 |
| 1.1746 | 2.9435 | 44600 | 1.1672 |
| 1.1373 | 2.9567 | 44800 | 1.1671 |
| 1.1532 | 2.9699 | 45000 | 1.1671 |
| 1.1256 | 2.9831 | 45200 | 1.1671 |
| 1.1338 | 2.9963 | 45400 | 1.1672 |
| 1.2044 | 3.0 | 45456 | 1.1672 |
Framework versions
- Transformers 5.14.1
- Pytorch 2.13.0+cu130
- Datasets 5.0.1
- Tokenizers 0.22.2