en_clm_child_30_new
This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 3.6211
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 30
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 40000
- training_steps: 100000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 1.3952 | 2000 | 6.8336 |
| 6.7315 | 2.7904 | 4000 | 5.2051 |
| 6.7315 | 4.1856 | 6000 | 4.7757 |
| 4.6005 | 5.5807 | 8000 | 4.5113 |
| 4.6005 | 6.9759 | 10000 | 4.3298 |
| 4.1342 | 8.3711 | 12000 | 4.1849 |
| 4.1342 | 9.7663 | 14000 | 4.0607 |
| 3.8587 | 11.1615 | 16000 | 3.9586 |
| 3.8587 | 12.5567 | 18000 | 3.8777 |
| 3.6556 | 13.9519 | 20000 | 3.7996 |
| 3.6556 | 15.3471 | 22000 | 3.7274 |
| 3.4935 | 16.7422 | 24000 | 3.6679 |
| 3.4935 | 18.1374 | 26000 | 3.6219 |
| 3.3583 | 19.5326 | 28000 | 3.5786 |
| 3.3583 | 20.9344 | 30000 | 3.5397 |
| 3.2472 | 22.3296 | 32000 | 3.5130 |
| 3.2472 | 23.7248 | 34000 | 3.4834 |
| 3.155 | 25.1200 | 36000 | 3.4656 |
| 3.155 | 26.5152 | 38000 | 3.4519 |
| 3.0735 | 27.9104 | 40000 | 3.4349 |
| 3.0735 | 29.3055 | 42000 | 3.4331 |
| 2.9918 | 30.7007 | 44000 | 3.4281 |
| 2.9918 | 32.0959 | 46000 | 3.4244 |
| 2.9142 | 33.4911 | 48000 | 3.4274 |
| 2.9142 | 34.8863 | 50000 | 3.4263 |
| 2.8463 | 36.2815 | 52000 | 3.4423 |
| 2.8463 | 37.6767 | 54000 | 3.4404 |
| 2.7874 | 39.0719 | 56000 | 3.4536 |
| 2.7874 | 40.4670 | 58000 | 3.4571 |
| 2.7308 | 41.8622 | 60000 | 3.4618 |
| 2.7308 | 43.2574 | 62000 | 3.4819 |
| 2.6794 | 44.6526 | 64000 | 3.4845 |
| 2.6794 | 46.0478 | 66000 | 3.4993 |
| 2.6359 | 47.4430 | 68000 | 3.5094 |
| 2.6359 | 48.8382 | 70000 | 3.5162 |
| 2.5965 | 50.2333 | 72000 | 3.5288 |
| 2.5965 | 51.6285 | 74000 | 3.5392 |
| 2.5601 | 53.0237 | 76000 | 3.5493 |
| 2.5601 | 54.4189 | 78000 | 3.5594 |
| 2.5232 | 55.8141 | 80000 | 3.5640 |
| 2.5232 | 57.2093 | 82000 | 3.5806 |
| 2.4919 | 58.6045 | 84000 | 3.5851 |
| 2.4919 | 59.9997 | 86000 | 3.5865 |
| 2.4662 | 61.3948 | 88000 | 3.6015 |
| 2.4662 | 62.7900 | 90000 | 3.6012 |
| 2.4409 | 64.1852 | 92000 | 3.6106 |
| 2.4409 | 65.5961 | 94000 | 3.6133 |
| 2.4194 | 66.9913 | 96000 | 3.6178 |
| 2.4194 | 68.3865 | 98000 | 3.6209 |
| 2.4004 | 69.7817 | 100000 | 3.6211 |
Framework versions
- Transformers 4.45.2
- Pytorch 2.5.1+cu124
- Datasets 3.0.1
- Tokenizers 0.20.1