trackmania-tmnf-gpt2-movevocab
This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.3897
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 6.0
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.9951 | 0.0279 | 200 | 1.8964 |
| 1.8955 | 0.0559 | 400 | 1.8377 |
| 1.8257 | 0.0838 | 600 | 1.8095 |
| 1.8104 | 0.1117 | 800 | 1.7616 |
| 1.7466 | 0.1397 | 1000 | 1.7344 |
| 1.7439 | 0.1676 | 1200 | 1.7141 |
| 1.6908 | 0.1955 | 1400 | 1.6942 |
| 1.7290 | 0.2235 | 1600 | 1.6766 |
| 1.6949 | 0.2514 | 1800 | 1.6629 |
| 1.6828 | 0.2793 | 2000 | 1.6483 |
| 1.6739 | 0.3073 | 2200 | 1.6413 |
| 1.6842 | 0.3352 | 2400 | 1.6289 |
| 1.6136 | 0.3631 | 2600 | 1.6214 |
| 1.6680 | 0.3911 | 2800 | 1.6166 |
| 1.6599 | 0.4190 | 3000 | 1.6053 |
| 1.6655 | 0.4469 | 3200 | 1.5987 |
| 1.5913 | 0.4749 | 3400 | 1.5909 |
| 1.6091 | 0.5028 | 3600 | 1.5865 |
| 1.6623 | 0.5307 | 3800 | 1.5774 |
| 1.6054 | 0.5587 | 4000 | 1.5695 |
| 1.6054 | 0.5866 | 4200 | 1.5674 |
| 1.6079 | 0.6145 | 4400 | 1.5622 |
| 1.6067 | 0.6425 | 4600 | 1.5551 |
| 1.5821 | 0.6704 | 4800 | 1.5537 |
| 1.5625 | 0.6983 | 5000 | 1.5504 |
| 1.5416 | 0.7263 | 5200 | 1.5426 |
| 1.5529 | 0.7542 | 5400 | 1.5382 |
| 1.5626 | 0.7821 | 5600 | 1.5342 |
| 1.6140 | 0.8101 | 5800 | 1.5302 |
| 1.5539 | 0.8380 | 6000 | 1.5275 |
| 1.5766 | 0.8659 | 6200 | 1.5240 |
| 1.5658 | 0.8939 | 6400 | 1.5177 |
| 1.5565 | 0.9218 | 6600 | 1.5133 |
| 1.5389 | 0.9497 | 6800 | 1.5143 |
| 1.5728 | 0.9777 | 7000 | 1.5092 |
| 1.5176 | 1.0056 | 7200 | 1.5063 |
| 1.5412 | 1.0335 | 7400 | 1.5034 |
| 1.5036 | 1.0615 | 7600 | 1.5004 |
| 1.4961 | 1.0894 | 7800 | 1.4982 |
| 1.5272 | 1.1173 | 8000 | 1.4974 |
| 1.5215 | 1.1453 | 8200 | 1.4925 |
| 1.5053 | 1.1732 | 8400 | 1.4901 |
| 1.5218 | 1.2011 | 8600 | 1.4874 |
| 1.4780 | 1.2291 | 8800 | 1.4860 |
| 1.5200 | 1.2570 | 9000 | 1.4839 |
| 1.4992 | 1.2849 | 9200 | 1.4817 |
| 1.4854 | 1.3128 | 9400 | 1.4791 |
| 1.4703 | 1.3408 | 9600 | 1.4791 |
| 1.4910 | 1.3687 | 9800 | 1.4755 |
| 1.4904 | 1.3966 | 10000 | 1.4721 |
| 1.4850 | 1.4246 | 10200 | 1.4714 |
| 1.5003 | 1.4525 | 10400 | 1.4700 |
| 1.5030 | 1.4804 | 10600 | 1.4680 |
| 1.5059 | 1.5084 | 10800 | 1.4654 |
| 1.4942 | 1.5363 | 11000 | 1.4647 |
| 1.4951 | 1.5642 | 11200 | 1.4631 |
| 1.4661 | 1.5922 | 11400 | 1.4614 |
| 1.4573 | 1.6201 | 11600 | 1.4601 |
| 1.4897 | 1.6480 | 11800 | 1.4599 |
| 1.4991 | 1.6760 | 12000 | 1.4575 |
| 1.4438 | 1.7039 | 12200 | 1.4558 |
| 1.4780 | 1.7318 | 12400 | 1.4559 |
| 1.4592 | 1.7598 | 12600 | 1.4560 |
| 1.4624 | 1.7877 | 12800 | 1.4524 |
| 1.4312 | 1.8156 | 13000 | 1.4510 |
| 1.4920 | 1.8436 | 13200 | 1.4500 |
| 1.4753 | 1.8715 | 13400 | 1.4474 |
| 1.4597 | 1.8994 | 13600 | 1.4476 |
| 1.4363 | 1.9274 | 13800 | 1.4447 |
| 1.4595 | 1.9553 | 14000 | 1.4462 |
| 1.4935 | 1.9832 | 14200 | 1.4437 |
| 1.4247 | 2.0112 | 14400 | 1.4447 |
| 1.4510 | 2.0391 | 14600 | 1.4432 |
| 1.4271 | 2.0670 | 14800 | 1.4408 |
| 1.4505 | 2.0950 | 15000 | 1.4398 |
| 1.4665 | 2.1229 | 15200 | 1.4404 |
| 1.4508 | 2.1508 | 15400 | 1.4375 |
| 1.4557 | 2.1788 | 15600 | 1.4383 |
| 1.4909 | 2.2067 | 15800 | 1.4353 |
| 1.4435 | 2.2346 | 16000 | 1.4362 |
| 1.4254 | 2.2626 | 16200 | 1.4350 |
| 1.4247 | 2.2905 | 16400 | 1.4337 |
| 1.4253 | 2.3184 | 16600 | 1.4329 |
| 1.4291 | 2.3464 | 16800 | 1.4314 |
| 1.4255 | 2.3743 | 17000 | 1.4309 |
| 1.4362 | 2.4022 | 17200 | 1.4316 |
| 1.4060 | 2.4302 | 17400 | 1.4305 |
| 1.4442 | 2.4581 | 17600 | 1.4297 |
| 1.4565 | 2.4860 | 17800 | 1.4270 |
| 1.4477 | 2.5140 | 18000 | 1.4286 |
| 1.4278 | 2.5419 | 18200 | 1.4280 |
| 1.4398 | 2.5698 | 18400 | 1.4244 |
| 1.4469 | 2.5978 | 18600 | 1.4259 |
| 1.4593 | 2.6257 | 18800 | 1.4228 |
| 1.4410 | 2.6536 | 19000 | 1.4252 |
| 1.4421 | 2.6816 | 19200 | 1.4227 |
| 1.4500 | 2.7095 | 19400 | 1.4224 |
| 1.4424 | 2.7374 | 19600 | 1.4213 |
| 1.4133 | 2.7654 | 19800 | 1.4215 |
| 1.4602 | 2.7933 | 20000 | 1.4222 |
| 1.4493 | 2.8212 | 20200 | 1.4198 |
| 1.4565 | 2.8492 | 20400 | 1.4193 |
| 1.4157 | 2.8771 | 20600 | 1.4184 |
| 1.4342 | 2.9050 | 20800 | 1.4174 |
| 1.4427 | 2.9330 | 21000 | 1.4177 |
| 1.4226 | 2.9609 | 21200 | 1.4172 |
| 1.4230 | 2.9888 | 21400 | 1.4166 |
| 1.3998 | 3.0168 | 21600 | 1.4164 |
| 1.4553 | 3.0447 | 21800 | 1.4144 |
| 1.3785 | 3.0726 | 22000 | 1.4149 |
| 1.4024 | 3.1006 | 22200 | 1.4155 |
| 1.4502 | 3.1285 | 22400 | 1.4138 |
| 1.4102 | 3.1564 | 22600 | 1.4145 |
| 1.4014 | 3.1844 | 22800 | 1.4138 |
| 1.4072 | 3.2123 | 23000 | 1.4142 |
| 1.3861 | 3.2402 | 23200 | 1.4137 |
| 1.4007 | 3.2682 | 23400 | 1.4124 |
| 1.4266 | 3.2961 | 23600 | 1.4098 |
| 1.3912 | 3.3240 | 23800 | 1.4108 |
| 1.3878 | 3.3520 | 24000 | 1.4112 |
| 1.4232 | 3.3799 | 24200 | 1.4083 |
| 1.4007 | 3.4078 | 24400 | 1.4102 |
| 1.4004 | 3.4358 | 24600 | 1.4083 |
| 1.4044 | 3.4637 | 24800 | 1.4085 |
| 1.4039 | 3.4916 | 25000 | 1.4084 |
| 1.4023 | 3.5196 | 25200 | 1.4090 |
| 1.3847 | 3.5475 | 25400 | 1.4061 |
| 1.4094 | 3.5754 | 25600 | 1.4075 |
| 1.4166 | 3.6034 | 25800 | 1.4064 |
| 1.4193 | 3.6313 | 26000 | 1.4060 |
| 1.4067 | 3.6592 | 26200 | 1.4052 |
| 1.4366 | 3.6872 | 26400 | 1.4052 |
| 1.4107 | 3.7151 | 26600 | 1.4053 |
| 1.3892 | 3.7430 | 26800 | 1.4044 |
| 1.3909 | 3.7709 | 27000 | 1.4033 |
| 1.3617 | 3.7989 | 27200 | 1.4047 |
| 1.3508 | 3.8268 | 27400 | 1.4039 |
| 1.4092 | 3.8547 | 27600 | 1.4032 |
| 1.4065 | 3.8827 | 27800 | 1.4025 |
| 1.3871 | 3.9106 | 28000 | 1.4019 |
| 1.3793 | 3.9385 | 28200 | 1.4025 |
| 1.3913 | 3.9665 | 28400 | 1.4007 |
| 1.3766 | 3.9944 | 28600 | 1.4014 |
| 1.4275 | 4.0223 | 28800 | 1.4023 |
| 1.4121 | 4.0503 | 29000 | 1.4010 |
| 1.4274 | 4.0782 | 29200 | 1.4015 |
| 1.4133 | 4.1061 | 29400 | 1.3998 |
| 1.3716 | 4.1341 | 29600 | 1.4012 |
| 1.4064 | 4.1620 | 29800 | 1.3998 |
| 1.3800 | 4.1899 | 30000 | 1.4002 |
| 1.3988 | 4.2179 | 30200 | 1.3998 |
| 1.4006 | 4.2458 | 30400 | 1.3987 |
| 1.3993 | 4.2737 | 30600 | 1.3991 |
| 1.3838 | 4.3017 | 30800 | 1.3985 |
| 1.3719 | 4.3296 | 31000 | 1.3993 |
| 1.3664 | 4.3575 | 31200 | 1.3990 |
| 1.3972 | 4.3855 | 31400 | 1.3979 |
| 1.4011 | 4.4134 | 31600 | 1.3970 |
| 1.3572 | 4.4413 | 31800 | 1.3975 |
| 1.4398 | 4.4693 | 32000 | 1.3963 |
| 1.3488 | 4.4972 | 32200 | 1.3987 |
| 1.3859 | 4.5251 | 32400 | 1.3970 |
| 1.3762 | 4.5531 | 32600 | 1.3967 |
| 1.4110 | 4.5810 | 32800 | 1.3950 |
| 1.3653 | 4.6089 | 33000 | 1.3946 |
| 1.3901 | 4.6369 | 33200 | 1.3958 |
| 1.3696 | 4.6648 | 33400 | 1.3950 |
| 1.3783 | 4.6927 | 33600 | 1.3941 |
| 1.3837 | 4.7207 | 33800 | 1.3946 |
| 1.3931 | 4.7486 | 34000 | 1.3937 |
| 1.3753 | 4.7765 | 34200 | 1.3948 |
| 1.3897 | 4.8045 | 34400 | 1.3932 |
| 1.3987 | 4.8324 | 34600 | 1.3932 |
| 1.3883 | 4.8603 | 34800 | 1.3933 |
| 1.3660 | 4.8883 | 35000 | 1.3935 |
| 1.3378 | 4.9162 | 35200 | 1.3930 |
| 1.4005 | 4.9441 | 35400 | 1.3934 |
| 1.3711 | 4.9721 | 35600 | 1.3935 |
| 1.3674 | 5.0 | 35800 | 1.3925 |
| 1.3640 | 5.0279 | 36000 | 1.3935 |
| 1.4034 | 5.0559 | 36200 | 1.3930 |
| 1.3911 | 5.0838 | 36400 | 1.3938 |
| 1.4070 | 5.1117 | 36600 | 1.3931 |
| 1.3642 | 5.1397 | 36800 | 1.3939 |
| 1.3664 | 5.1676 | 37000 | 1.3928 |
| 1.3831 | 5.1955 | 37200 | 1.3927 |
| 1.3725 | 5.2235 | 37400 | 1.3924 |
| 1.3767 | 5.2514 | 37600 | 1.3923 |
| 1.3554 | 5.2793 | 37800 | 1.3922 |
| 1.3728 | 5.3073 | 38000 | 1.3918 |
| 1.3883 | 5.3352 | 38200 | 1.3919 |
| 1.3919 | 5.3631 | 38400 | 1.3918 |
| 1.3946 | 5.3911 | 38600 | 1.3913 |
| 1.3579 | 5.4190 | 38800 | 1.3911 |
| 1.3512 | 5.4469 | 39000 | 1.3912 |
| 1.4137 | 5.4749 | 39200 | 1.3905 |
| 1.3594 | 5.5028 | 39400 | 1.3907 |
| 1.3535 | 5.5307 | 39600 | 1.3911 |
| 1.3501 | 5.5587 | 39800 | 1.3907 |
| 1.4001 | 5.5866 | 40000 | 1.3903 |
| 1.3673 | 5.6145 | 40200 | 1.3904 |
| 1.3572 | 5.6425 | 40400 | 1.3906 |
| 1.3675 | 5.6704 | 40600 | 1.3906 |
| 1.3946 | 5.6983 | 40800 | 1.3901 |
| 1.3347 | 5.7263 | 41000 | 1.3903 |
| 1.3774 | 5.7542 | 41200 | 1.3903 |
| 1.3693 | 5.7821 | 41400 | 1.3899 |
| 1.3830 | 5.8101 | 41600 | 1.3901 |
| 1.3719 | 5.8380 | 41800 | 1.3898 |
| 1.3861 | 5.8659 | 42000 | 1.3899 |
| 1.3622 | 5.8939 | 42200 | 1.3896 |
| 1.3664 | 5.9218 | 42400 | 1.3897 |
| 1.3676 | 5.9497 | 42600 | 1.3895 |
| 1.3721 | 5.9777 | 42800 | 1.3897 |
| 1.3866 | 6.0 | 42960 | 1.3897 |
Framework versions
- Transformers 5.14.1
- Pytorch 2.13.0+cu130
- Datasets 5.0.1
- Tokenizers 0.22.2