recipe-distilroberta-Is
This model is a fine-tuned version of distilroberta-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 4.7427
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 25
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 19.6191 | 1.0 | 2135 | 10.5217 |
| 8.6838 | 2.0 | 4270 | 7.3017 |
| 6.884 | 3.0 | 6405 | 6.4445 |
| 6.2953 | 4.0 | 8540 | 6.0610 |
| 6.0205 | 5.0 | 10675 | 5.9047 |
| 5.851 | 6.0 | 12810 | 5.7790 |
| 5.7464 | 7.0 | 14945 | 5.7164 |
| 5.6684 | 8.0 | 17080 | 5.6415 |
| 5.6138 | 9.0 | 19215 | 5.5671 |
| 5.5638 | 10.0 | 21350 | 5.5360 |
| 5.5288 | 11.0 | 23485 | 5.5069 |
| 5.4968 | 12.0 | 25620 | 5.4968 |
| 5.4696 | 13.0 | 27755 | 5.4539 |
| 5.4468 | 14.0 | 29890 | 5.4416 |
| 5.4177 | 15.0 | 32025 | 5.3722 |
| 5.3717 | 16.0 | 34160 | 5.3226 |
| 5.317 | 17.0 | 36295 | 5.2197 |
| 5.2367 | 18.0 | 38430 | 5.0888 |
| 5.1543 | 19.0 | 40565 | 4.9954 |
| 5.0919 | 20.0 | 42700 | 4.9306 |
| 5.038 | 21.0 | 44835 | 4.8657 |
| 4.9983 | 22.0 | 46970 | 4.8137 |
| 4.9639 | 23.0 | 49105 | 4.7704 |
| 4.9426 | 24.0 | 51240 | 4.7486 |
| 4.9312 | 25.0 | 53375 | 4.7427 |
Framework versions
- Transformers 4.19.0.dev0
- Pytorch 1.9.0+cu111
- Datasets 2.4.0
- Tokenizers 0.12.1