trackmania-tmnf-gpt2-safefilter
This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.1939
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3.0
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.7056 | 0.0130 | 200 | 1.6578 |
| 1.5915 | 0.0260 | 400 | 1.5942 |
| 1.5906 | 0.0389 | 600 | 1.5522 |
| 1.5400 | 0.0519 | 800 | 1.5102 |
| 1.4781 | 0.0649 | 1000 | 1.4844 |
| 1.4849 | 0.0779 | 1200 | 1.4539 |
| 1.4936 | 0.0909 | 1400 | 1.4334 |
| 1.4454 | 0.1038 | 1600 | 1.4192 |
| 1.4749 | 0.1168 | 1800 | 1.4015 |
| 1.4100 | 0.1298 | 2000 | 1.4019 |
| 1.4486 | 0.1428 | 2200 | 1.3842 |
| 1.4191 | 0.1558 | 2400 | 1.3724 |
| 1.3738 | 0.1687 | 2600 | 1.3669 |
| 1.4334 | 0.1817 | 2800 | 1.3577 |
| 1.3914 | 0.1947 | 3000 | 1.3505 |
| 1.3810 | 0.2077 | 3200 | 1.3466 |
| 1.3597 | 0.2207 | 3400 | 1.3412 |
| 1.3526 | 0.2336 | 3600 | 1.3375 |
| 1.3493 | 0.2466 | 3800 | 1.3335 |
| 1.3912 | 0.2596 | 4000 | 1.3275 |
| 1.3681 | 0.2726 | 4200 | 1.3229 |
| 1.3494 | 0.2855 | 4400 | 1.3183 |
| 1.3535 | 0.2985 | 4600 | 1.3164 |
| 1.3243 | 0.3115 | 4800 | 1.3109 |
| 1.3057 | 0.3245 | 5000 | 1.3093 |
| 1.3310 | 0.3375 | 5200 | 1.3076 |
| 1.3608 | 0.3504 | 5400 | 1.3060 |
| 1.3252 | 0.3634 | 5600 | 1.3028 |
| 1.3465 | 0.3764 | 5800 | 1.2986 |
| 1.3504 | 0.3894 | 6000 | 1.2991 |
| 1.3122 | 0.4024 | 6200 | 1.2936 |
| 1.3321 | 0.4153 | 6400 | 1.2967 |
| 1.2897 | 0.4283 | 6600 | 1.2898 |
| 1.2857 | 0.4413 | 6800 | 1.2884 |
| 1.3037 | 0.4543 | 7000 | 1.2898 |
| 1.3315 | 0.4673 | 7200 | 1.2850 |
| 1.3149 | 0.4802 | 7400 | 1.2848 |
| 1.2690 | 0.4932 | 7600 | 1.2841 |
| 1.3165 | 0.5062 | 7800 | 1.2767 |
| 1.3232 | 0.5192 | 8000 | 1.2773 |
| 1.3642 | 0.5322 | 8200 | 1.2756 |
| 1.3153 | 0.5451 | 8400 | 1.2738 |
| 1.2903 | 0.5581 | 8600 | 1.2722 |
| 1.3178 | 0.5711 | 8800 | 1.2728 |
| 1.3113 | 0.5841 | 9000 | 1.2692 |
| 1.2718 | 0.5971 | 9200 | 1.2686 |
| 1.3060 | 0.6100 | 9400 | 1.2680 |
| 1.2423 | 0.6230 | 9600 | 1.2658 |
| 1.2594 | 0.6360 | 9800 | 1.2656 |
| 1.2804 | 0.6490 | 10000 | 1.2636 |
| 1.3237 | 0.6620 | 10200 | 1.2621 |
| 1.2723 | 0.6749 | 10400 | 1.2611 |
| 1.2613 | 0.6879 | 10600 | 1.2628 |
| 1.2405 | 0.7009 | 10800 | 1.2618 |
| 1.2204 | 0.7139 | 11000 | 1.2590 |
| 1.2510 | 0.7268 | 11200 | 1.2578 |
| 1.2509 | 0.7398 | 11400 | 1.2577 |
| 1.2306 | 0.7528 | 11600 | 1.2554 |
| 1.2825 | 0.7658 | 11800 | 1.2522 |
| 1.2035 | 0.7788 | 12000 | 1.2534 |
| 1.3008 | 0.7917 | 12200 | 1.2519 |
| 1.2703 | 0.8047 | 12400 | 1.2506 |
| 1.2355 | 0.8177 | 12600 | 1.2503 |
| 1.2895 | 0.8307 | 12800 | 1.2486 |
| 1.2276 | 0.8437 | 13000 | 1.2486 |
| 1.2523 | 0.8566 | 13200 | 1.2464 |
| 1.3092 | 0.8696 | 13400 | 1.2465 |
| 1.2665 | 0.8826 | 13600 | 1.2459 |
| 1.2557 | 0.8956 | 13800 | 1.2465 |
| 1.2371 | 0.9086 | 14000 | 1.2424 |
| 1.2513 | 0.9215 | 14200 | 1.2427 |
| 1.2290 | 0.9345 | 14400 | 1.2433 |
| 1.2591 | 0.9475 | 14600 | 1.2430 |
| 1.2164 | 0.9605 | 14800 | 1.2407 |
| 1.2448 | 0.9735 | 15000 | 1.2430 |
| 1.2457 | 0.9864 | 15200 | 1.2402 |
| 1.2892 | 0.9994 | 15400 | 1.2393 |
| 1.2484 | 1.0124 | 15600 | 1.2384 |
| 1.2266 | 1.0254 | 15800 | 1.2388 |
| 1.2149 | 1.0384 | 16000 | 1.2361 |
| 1.2330 | 1.0513 | 16200 | 1.2344 |
| 1.2542 | 1.0643 | 16400 | 1.2358 |
| 1.2738 | 1.0773 | 16600 | 1.2352 |
| 1.2572 | 1.0903 | 16800 | 1.2329 |
| 1.2383 | 1.1033 | 17000 | 1.2356 |
| 1.2365 | 1.1162 | 17200 | 1.2317 |
| 1.2841 | 1.1292 | 17400 | 1.2313 |
| 1.2130 | 1.1422 | 17600 | 1.2332 |
| 1.2151 | 1.1552 | 17800 | 1.2317 |
| 1.2047 | 1.1681 | 18000 | 1.2288 |
| 1.2456 | 1.1811 | 18200 | 1.2290 |
| 1.3059 | 1.1941 | 18400 | 1.2295 |
| 1.2084 | 1.2071 | 18600 | 1.2285 |
| 1.2633 | 1.2201 | 18800 | 1.2278 |
| 1.2259 | 1.2330 | 19000 | 1.2271 |
| 1.2305 | 1.2460 | 19200 | 1.2256 |
| 1.2864 | 1.2590 | 19400 | 1.2258 |
| 1.2274 | 1.2720 | 19600 | 1.2269 |
| 1.2991 | 1.2850 | 19800 | 1.2237 |
| 1.2292 | 1.2979 | 20000 | 1.2241 |
| 1.2019 | 1.3109 | 20200 | 1.2236 |
| 1.2050 | 1.3239 | 20400 | 1.2249 |
| 1.2206 | 1.3369 | 20600 | 1.2228 |
| 1.2121 | 1.3499 | 20800 | 1.2229 |
| 1.2152 | 1.3628 | 21000 | 1.2224 |
| 1.2369 | 1.3758 | 21200 | 1.2211 |
| 1.2167 | 1.3888 | 21400 | 1.2208 |
| 1.1991 | 1.4018 | 21600 | 1.2196 |
| 1.3107 | 1.4148 | 21800 | 1.2193 |
| 1.2649 | 1.4277 | 22000 | 1.2201 |
| 1.2421 | 1.4407 | 22200 | 1.2192 |
| 1.2327 | 1.4537 | 22400 | 1.2190 |
| 1.1790 | 1.4667 | 22600 | 1.2199 |
| 1.2614 | 1.4797 | 22800 | 1.2184 |
| 1.2354 | 1.4926 | 23000 | 1.2163 |
| 1.2326 | 1.5056 | 23200 | 1.2162 |
| 1.2342 | 1.5186 | 23400 | 1.2176 |
| 1.1749 | 1.5316 | 23600 | 1.2173 |
| 1.2464 | 1.5446 | 23800 | 1.2153 |
| 1.2146 | 1.5575 | 24000 | 1.2156 |
| 1.2696 | 1.5705 | 24200 | 1.2141 |
| 1.2319 | 1.5835 | 24400 | 1.2162 |
| 1.2167 | 1.5965 | 24600 | 1.2134 |
| 1.2653 | 1.6094 | 24800 | 1.2147 |
| 1.2424 | 1.6224 | 25000 | 1.2134 |
| 1.1754 | 1.6354 | 25200 | 1.2128 |
| 1.2197 | 1.6484 | 25400 | 1.2124 |
| 1.2092 | 1.6614 | 25600 | 1.2125 |
| 1.2013 | 1.6743 | 25800 | 1.2119 |
| 1.1494 | 1.6873 | 26000 | 1.2124 |
| 1.1959 | 1.7003 | 26200 | 1.2116 |
| 1.2310 | 1.7133 | 26400 | 1.2108 |
| 1.2113 | 1.7263 | 26600 | 1.2111 |
| 1.2529 | 1.7392 | 26800 | 1.2100 |
| 1.1876 | 1.7522 | 27000 | 1.2116 |
| 1.2514 | 1.7652 | 27200 | 1.2108 |
| 1.1956 | 1.7782 | 27400 | 1.2117 |
| 1.1787 | 1.7912 | 27600 | 1.2108 |
| 1.1680 | 1.8041 | 27800 | 1.2096 |
| 1.2124 | 1.8171 | 28000 | 1.2101 |
| 1.1817 | 1.8301 | 28200 | 1.2074 |
| 1.2394 | 1.8431 | 28400 | 1.2088 |
| 1.2403 | 1.8561 | 28600 | 1.2074 |
| 1.2057 | 1.8690 | 28800 | 1.2072 |
| 1.1999 | 1.8820 | 29000 | 1.2078 |
| 1.2519 | 1.8950 | 29200 | 1.2066 |
| 1.1860 | 1.9080 | 29400 | 1.2070 |
| 1.2699 | 1.9210 | 29600 | 1.2046 |
| 1.2602 | 1.9339 | 29800 | 1.2058 |
| 1.2701 | 1.9469 | 30000 | 1.2058 |
| 1.1598 | 1.9599 | 30200 | 1.2061 |
| 1.2530 | 1.9729 | 30400 | 1.2060 |
| 1.1839 | 1.9859 | 30600 | 1.2043 |
| 1.2322 | 1.9988 | 30800 | 1.2050 |
| 1.2350 | 2.0118 | 31000 | 1.2034 |
| 1.2441 | 2.0248 | 31200 | 1.2043 |
| 1.2328 | 2.0378 | 31400 | 1.2044 |
| 1.1829 | 2.0507 | 31600 | 1.2045 |
| 1.2101 | 2.0637 | 31800 | 1.2056 |
| 1.2069 | 2.0767 | 32000 | 1.2032 |
| 1.2969 | 2.0897 | 32200 | 1.2021 |
| 1.2198 | 2.1027 | 32400 | 1.2031 |
| 1.1780 | 2.1156 | 32600 | 1.2035 |
| 1.2017 | 2.1286 | 32800 | 1.2014 |
| 1.2250 | 2.1416 | 33000 | 1.2028 |
| 1.2172 | 2.1546 | 33200 | 1.2024 |
| 1.1994 | 2.1676 | 33400 | 1.2016 |
| 1.2212 | 2.1805 | 33600 | 1.2013 |
| 1.1866 | 2.1935 | 33800 | 1.2015 |
| 1.1548 | 2.2065 | 34000 | 1.2016 |
| 1.2074 | 2.2195 | 34200 | 1.2007 |
| 1.1574 | 2.2325 | 34400 | 1.2016 |
| 1.1578 | 2.2454 | 34600 | 1.2018 |
| 1.2136 | 2.2584 | 34800 | 1.2009 |
| 1.1522 | 2.2714 | 35000 | 1.2004 |
| 1.1858 | 2.2844 | 35200 | 1.2004 |
| 1.2059 | 2.2974 | 35400 | 1.2003 |
| 1.1552 | 2.3103 | 35600 | 1.1991 |
| 1.1685 | 2.3233 | 35800 | 1.1992 |
| 1.2464 | 2.3363 | 36000 | 1.1986 |
| 1.2305 | 2.3493 | 36200 | 1.1994 |
| 1.1945 | 2.3623 | 36400 | 1.1995 |
| 1.2013 | 2.3752 | 36600 | 1.1984 |
| 1.2105 | 2.3882 | 36800 | 1.1981 |
| 1.1943 | 2.4012 | 37000 | 1.1991 |
| 1.2376 | 2.4142 | 37200 | 1.1979 |
| 1.1938 | 2.4272 | 37400 | 1.1980 |
| 1.2158 | 2.4401 | 37600 | 1.1977 |
| 1.1661 | 2.4531 | 37800 | 1.1993 |
| 1.2016 | 2.4661 | 38000 | 1.1973 |
| 1.2250 | 2.4791 | 38200 | 1.1977 |
| 1.1726 | 2.4921 | 38400 | 1.1970 |
| 1.2189 | 2.5050 | 38600 | 1.1978 |
| 1.1787 | 2.5180 | 38800 | 1.1970 |
| 1.1772 | 2.5310 | 39000 | 1.1971 |
| 1.2353 | 2.5440 | 39200 | 1.1968 |
| 1.1860 | 2.5569 | 39400 | 1.1964 |
| 1.1603 | 2.5699 | 39600 | 1.1966 |
| 1.2319 | 2.5829 | 39800 | 1.1952 |
| 1.1659 | 2.5959 | 40000 | 1.1967 |
| 1.2017 | 2.6089 | 40200 | 1.1967 |
| 1.1795 | 2.6218 | 40400 | 1.1956 |
| 1.1689 | 2.6348 | 40600 | 1.1964 |
| 1.2023 | 2.6478 | 40800 | 1.1959 |
| 1.2229 | 2.6608 | 41000 | 1.1954 |
| 1.1617 | 2.6738 | 41200 | 1.1961 |
| 1.2204 | 2.6867 | 41400 | 1.1952 |
| 1.1743 | 2.6997 | 41600 | 1.1956 |
| 1.1798 | 2.7127 | 41800 | 1.1950 |
| 1.2255 | 2.7257 | 42000 | 1.1959 |
| 1.1755 | 2.7387 | 42200 | 1.1949 |
| 1.2212 | 2.7516 | 42400 | 1.1949 |
| 1.2049 | 2.7646 | 42600 | 1.1948 |
| 1.1908 | 2.7776 | 42800 | 1.1947 |
| 1.1742 | 2.7906 | 43000 | 1.1948 |
| 1.1913 | 2.8036 | 43200 | 1.1944 |
| 1.2022 | 2.8165 | 43400 | 1.1946 |
| 1.1924 | 2.8295 | 43600 | 1.1949 |
| 1.2167 | 2.8425 | 43800 | 1.1946 |
| 1.1931 | 2.8555 | 44000 | 1.1943 |
| 1.2242 | 2.8685 | 44200 | 1.1942 |
| 1.2159 | 2.8814 | 44400 | 1.1937 |
| 1.1917 | 2.8944 | 44600 | 1.1940 |
| 1.2102 | 2.9074 | 44800 | 1.1940 |
| 1.1772 | 2.9204 | 45000 | 1.1939 |
| 1.2313 | 2.9334 | 45200 | 1.1938 |
| 1.2232 | 2.9463 | 45400 | 1.1940 |
| 1.2114 | 2.9593 | 45600 | 1.1937 |
| 1.2056 | 2.9723 | 45800 | 1.1938 |
| 1.1758 | 2.9853 | 46000 | 1.1939 |
Framework versions
- Transformers 5.14.1
- Pytorch 2.13.0+cu130
- Datasets 5.0.1
- Tokenizers 0.22.2