Llama-3.3-70B-Instruct-3d-1M-100K-0.2-reverse-padzero-plus-mul-sub-99-512D-1L-2H-2048I
This model is a fine-tuned version of meta-llama/Llama-3.3-70B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.4081
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 0 | 0 | 3.1271 |
| 1.7391 | 0.0640 | 500 | 1.6688 |
| 1.5536 | 0.1280 | 1000 | 1.5245 |
| 1.4966 | 0.1920 | 1500 | 1.5046 |
| 1.4812 | 0.2560 | 2000 | 1.4866 |
| 1.4766 | 0.3200 | 2500 | 1.4725 |
| 1.4714 | 0.3840 | 3000 | 1.4746 |
| 1.4639 | 0.4480 | 3500 | 1.4657 |
| 1.4646 | 0.5120 | 4000 | 1.4685 |
| 1.455 | 0.5760 | 4500 | 1.4598 |
| 1.4554 | 0.6400 | 5000 | 1.4550 |
| 1.4561 | 0.7040 | 5500 | 1.4534 |
| 1.4536 | 0.7680 | 6000 | 1.4515 |
| 1.4523 | 0.8319 | 6500 | 1.4515 |
| 1.447 | 0.8959 | 7000 | 1.4507 |
| 1.45 | 0.9599 | 7500 | 1.4511 |
| 1.4466 | 1.0239 | 8000 | 1.4463 |
| 1.4457 | 1.0879 | 8500 | 1.4476 |
| 1.4407 | 1.1519 | 9000 | 1.4409 |
| 1.4377 | 1.2159 | 9500 | 1.4388 |
| 1.4369 | 1.2799 | 10000 | 1.4381 |
| 1.4352 | 1.3439 | 10500 | 1.4358 |
| 1.4339 | 1.4079 | 11000 | 1.4348 |
| 1.4345 | 1.4719 | 11500 | 1.4334 |
| 1.4319 | 1.5359 | 12000 | 1.4317 |
| 1.43 | 1.5999 | 12500 | 1.4342 |
| 1.4318 | 1.6639 | 13000 | 1.4296 |
| 1.429 | 1.7279 | 13500 | 1.4310 |
| 1.4277 | 1.7919 | 14000 | 1.4295 |
| 1.4292 | 1.8559 | 14500 | 1.4287 |
| 1.4286 | 1.9199 | 15000 | 1.4290 |
| 1.4248 | 1.9839 | 15500 | 1.4278 |
| 1.4282 | 2.0479 | 16000 | 1.4278 |
| 1.428 | 2.1119 | 16500 | 1.4300 |
| 1.4261 | 2.1759 | 17000 | 1.4252 |
| 1.426 | 2.2399 | 17500 | 1.4250 |
| 1.4259 | 2.3039 | 18000 | 1.4282 |
| 1.4241 | 2.3678 | 18500 | 1.4269 |
| 1.4249 | 2.4318 | 19000 | 1.4237 |
| 1.4234 | 2.4958 | 19500 | 1.4236 |
| 1.4229 | 2.5598 | 20000 | 1.4221 |
| 1.4202 | 2.6238 | 20500 | 1.4223 |
| 1.421 | 2.6878 | 21000 | 1.4213 |
| 1.4189 | 2.7518 | 21500 | 1.4221 |
| 1.4194 | 2.8158 | 22000 | 1.4200 |
| 1.42 | 2.8798 | 22500 | 1.4202 |
| 1.419 | 2.9438 | 23000 | 1.4192 |
| 1.4177 | 3.0078 | 23500 | 1.4177 |
| 1.4172 | 3.0718 | 24000 | 1.4154 |
| 1.4132 | 3.1358 | 24500 | 1.4148 |
| 1.4149 | 3.1998 | 25000 | 1.4132 |
| 1.4124 | 3.2638 | 25500 | 1.4131 |
| 1.4114 | 3.3278 | 26000 | 1.4123 |
| 1.4113 | 3.3918 | 26500 | 1.4119 |
| 1.4117 | 3.4558 | 27000 | 1.4111 |
| 1.4122 | 3.5198 | 27500 | 1.4103 |
| 1.4091 | 3.5838 | 28000 | 1.4103 |
| 1.4098 | 3.6478 | 28500 | 1.4101 |
| 1.4093 | 3.7118 | 29000 | 1.4096 |
| 1.4092 | 3.7758 | 29500 | 1.4092 |
| 1.408 | 3.8398 | 30000 | 1.4091 |
| 1.4115 | 3.9038 | 30500 | 1.4092 |
| 1.4098 | 3.9677 | 31000 | 1.4087 |
| 1.4104 | 4.0317 | 31500 | 1.4088 |
| 1.41 | 4.0957 | 32000 | 1.4086 |
| 1.4066 | 4.1597 | 32500 | 1.4084 |
| 1.4066 | 4.2237 | 33000 | 1.4084 |
| 1.4073 | 4.2877 | 33500 | 1.4083 |
| 1.4078 | 4.3517 | 34000 | 1.4082 |
| 1.4077 | 4.4157 | 34500 | 1.4083 |
| 1.4089 | 4.4797 | 35000 | 1.4082 |
| 1.4084 | 4.5437 | 35500 | 1.4082 |
| 1.4085 | 4.6077 | 36000 | 1.4081 |
| 1.4072 | 4.6717 | 36500 | 1.4081 |
| 1.4073 | 4.7357 | 37000 | 1.4081 |
| 1.4073 | 4.7997 | 37500 | 1.4081 |
| 1.4089 | 4.8637 | 38000 | 1.4081 |
| 1.4089 | 4.9277 | 38500 | 1.4081 |
| 1.4069 | 4.9917 | 39000 | 1.4081 |
Framework versions
- Transformers 4.57.1
- Pytorch 2.9.0+cu128
- Datasets 4.5.0
- Tokenizers 0.22.1