hypa-tiny-keys-2026-09-20_16-51-04
This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.1709
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 128
- eval_batch_size: 128
- seed: 3407
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- training_steps: 270523
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.2299 | 0.0025 | 677 | 2.4989 |
| 2.1747 | 0.0050 | 1354 | 1.8787 |
| 1.7638 | 0.0075 | 2031 | 1.6614 |
| 1.6165 | 0.0100 | 2708 | 1.5619 |
| 1.5382 | 0.0125 | 3385 | 1.5013 |
| 1.4862 | 0.0150 | 4062 | 1.4591 |
| 1.4479 | 0.0175 | 4739 | 1.4250 |
| 1.4335 | 0.0200 | 5416 | 1.4076 |
| 1.4134 | 0.0225 | 6093 | 1.3957 |
| 1.3858 | 0.0250 | 6770 | 1.3844 |
| 1.3631 | 0.0275 | 7447 | 1.3718 |
| 1.3448 | 0.0300 | 8124 | 1.3590 |
| 1.3291 | 0.0325 | 8801 | 1.3527 |
| 1.3141 | 0.0350 | 9478 | 1.3434 |
| 1.3053 | 0.0375 | 10155 | 1.3281 |
| 1.3269 | 0.0400 | 10832 | 1.3213 |
| 1.3058 | 0.0425 | 11509 | 1.3124 |
| 1.2912 | 0.0450 | 12186 | 1.3085 |
| 1.2792 | 0.0475 | 12863 | 1.3021 |
| 1.2683 | 0.0501 | 13540 | 1.2940 |
| 1.2588 | 0.0526 | 14217 | 1.2935 |
| 1.2508 | 0.0551 | 14894 | 1.2875 |
| 1.2657 | 0.0576 | 15571 | 1.2791 |
| 1.2640 | 0.0601 | 16248 | 1.2743 |
| 1.2524 | 0.0626 | 16925 | 1.2712 |
| 1.2412 | 0.0651 | 17602 | 1.2692 |
| 1.2326 | 0.0676 | 18279 | 1.2655 |
| 1.2258 | 0.0701 | 18956 | 1.2634 |
| 1.2180 | 0.0726 | 19633 | 1.2594 |
| 1.2179 | 0.0751 | 20310 | 1.2532 |
| 1.2418 | 0.0776 | 20987 | 1.2457 |
| 1.2256 | 0.0801 | 21664 | 1.2412 |
| 1.2162 | 0.0826 | 22341 | 1.2402 |
| 1.2077 | 0.0851 | 23018 | 1.2373 |
| 1.2009 | 0.0876 | 23695 | 1.2369 |
| 1.1941 | 0.0901 | 24372 | 1.2340 |
| 1.1914 | 0.0926 | 25049 | 1.2294 |
| 1.2115 | 0.0951 | 25726 | 1.2187 |
| 1.2084 | 0.0976 | 26403 | 1.2202 |
| 1.1973 | 0.1001 | 27080 | 1.2223 |
| 1.1899 | 0.1026 | 27757 | 1.2180 |
| 1.1847 | 0.1051 | 28434 | 1.2177 |
| 1.1784 | 0.1076 | 29111 | 1.2136 |
| 1.1723 | 0.1101 | 29788 | 1.2134 |
| 1.1774 | 0.1126 | 30465 | 1.2084 |
| 1.1983 | 0.1151 | 31142 | 1.2061 |
| 1.1849 | 0.1176 | 31819 | 1.2046 |
| 1.1763 | 0.1201 | 32496 | 1.2018 |
| 1.1708 | 0.1226 | 33173 | 1.2036 |
| 1.1655 | 0.1251 | 33850 | 1.2007 |
| 1.1613 | 0.1276 | 34527 | 1.2006 |
| 1.1557 | 0.1301 | 35204 | 1.2011 |
| 1.1827 | 0.1326 | 35881 | 1.1901 |
| 1.1769 | 0.1351 | 36558 | 1.1947 |
| 1.1661 | 0.1376 | 37235 | 1.1888 |
| 1.1592 | 0.1401 | 37912 | 1.1899 |
| 1.1546 | 0.1426 | 38589 | 1.1887 |
| 1.1507 | 0.1451 | 39266 | 1.1880 |
| 1.1454 | 0.1477 | 39943 | 1.1890 |
| 1.1526 | 0.1502 | 40620 | 1.1858 |
| 1.1730 | 0.1527 | 41297 | 1.1794 |
| 1.1596 | 0.1552 | 41974 | 1.1793 |
| 1.1517 | 0.1577 | 42651 | 1.1800 |
| 1.1469 | 0.1602 | 43328 | 1.1769 |
| 1.1428 | 0.1627 | 44005 | 1.1768 |
| 1.1384 | 0.1652 | 44682 | 1.1750 |
| 1.1361 | 0.1677 | 45359 | 1.1752 |
| 1.1622 | 0.1702 | 46036 | 1.1708 |
| 1.1551 | 0.1727 | 46713 | 1.1700 |
| 1.1472 | 0.1752 | 47390 | 1.1715 |
| 1.1400 | 0.1777 | 48067 | 1.1714 |
| 1.1362 | 0.1802 | 48744 | 1.1775 |
| 1.1315 | 0.1827 | 49421 | 1.1709 |
Framework versions
- Transformers 5.17.0
- Pytorch 2.11.0+cu128
- Datasets 5.0.1
- Tokenizers 0.23.2