Llama-3.3-70B-Instruct-3d-1M-100K-0.2-reverse-padzero-plus-mul-sub-99-64D-1L-2H-256I
This model is a fine-tuned version of meta-llama/Llama-3.3-70B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.4496
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 0 | 0 | 3.0497 |
| 2.1115 | 0.0640 | 500 | 1.9380 |
| 1.7864 | 0.1280 | 1000 | 1.7813 |
| 1.7188 | 0.1920 | 1500 | 1.6993 |
| 1.6056 | 0.2560 | 2000 | 1.6032 |
| 1.5877 | 0.3200 | 2500 | 1.5829 |
| 1.5754 | 0.3840 | 3000 | 1.5752 |
| 1.5662 | 0.4480 | 3500 | 1.5627 |
| 1.5588 | 0.5120 | 4000 | 1.5578 |
| 1.5542 | 0.5760 | 4500 | 1.5546 |
| 1.5408 | 0.6400 | 5000 | 1.5280 |
| 1.4913 | 0.7040 | 5500 | 1.4858 |
| 1.4793 | 0.7680 | 6000 | 1.4802 |
| 1.4761 | 0.8319 | 6500 | 1.4771 |
| 1.4722 | 0.8959 | 7000 | 1.4720 |
| 1.4697 | 0.9599 | 7500 | 1.4678 |
| 1.4664 | 1.0239 | 8000 | 1.4677 |
| 1.4703 | 1.0879 | 8500 | 1.4701 |
| 1.4652 | 1.1519 | 9000 | 1.4644 |
| 1.4652 | 1.2159 | 9500 | 1.4627 |
| 1.4622 | 1.2799 | 10000 | 1.4620 |
| 1.4624 | 1.3439 | 10500 | 1.4605 |
| 1.4613 | 1.4079 | 11000 | 1.4607 |
| 1.462 | 1.4719 | 11500 | 1.4588 |
| 1.4594 | 1.5359 | 12000 | 1.4589 |
| 1.4573 | 1.5999 | 12500 | 1.4595 |
| 1.4595 | 1.6639 | 13000 | 1.4584 |
| 1.4576 | 1.7279 | 13500 | 1.4577 |
| 1.4578 | 1.7919 | 14000 | 1.4659 |
| 1.4593 | 1.8559 | 14500 | 1.4562 |
| 1.4582 | 1.9199 | 15000 | 1.4601 |
| 1.4553 | 1.9839 | 15500 | 1.4577 |
| 1.458 | 2.0479 | 16000 | 1.4563 |
| 1.456 | 2.1119 | 16500 | 1.4545 |
| 1.4559 | 2.1759 | 17000 | 1.4537 |
| 1.4548 | 2.2399 | 17500 | 1.4536 |
| 1.4562 | 2.3039 | 18000 | 1.4546 |
| 1.4537 | 2.3678 | 18500 | 1.4531 |
| 1.4546 | 2.4318 | 19000 | 1.4538 |
| 1.4572 | 2.4958 | 19500 | 1.4569 |
| 1.457 | 2.5598 | 20000 | 1.4590 |
| 1.4526 | 2.6238 | 20500 | 1.4617 |
| 1.4525 | 2.6878 | 21000 | 1.4555 |
| 1.4515 | 2.7518 | 21500 | 1.4565 |
| 1.4519 | 2.8158 | 22000 | 1.4518 |
| 1.4538 | 2.8798 | 22500 | 1.4522 |
| 1.4517 | 2.9438 | 23000 | 1.4514 |
| 1.452 | 3.0078 | 23500 | 1.4512 |
| 1.4527 | 3.0718 | 24000 | 1.4513 |
| 1.4503 | 3.1358 | 24500 | 1.4516 |
| 1.4524 | 3.1998 | 25000 | 1.4519 |
| 1.4509 | 3.2638 | 25500 | 1.4508 |
| 1.4507 | 3.3278 | 26000 | 1.4504 |
| 1.451 | 3.3918 | 26500 | 1.4507 |
| 1.4501 | 3.4558 | 27000 | 1.4505 |
| 1.4513 | 3.5198 | 27500 | 1.4501 |
| 1.4493 | 3.5838 | 28000 | 1.4501 |
| 1.4504 | 3.6478 | 28500 | 1.4502 |
| 1.449 | 3.7118 | 29000 | 1.4499 |
| 1.4499 | 3.7758 | 29500 | 1.4498 |
| 1.4489 | 3.8398 | 30000 | 1.4500 |
| 1.4522 | 3.9038 | 30500 | 1.4498 |
| 1.4508 | 3.9677 | 31000 | 1.4497 |
| 1.4519 | 4.0317 | 31500 | 1.4497 |
| 1.4511 | 4.0957 | 32000 | 1.4497 |
| 1.4484 | 4.1597 | 32500 | 1.4497 |
| 1.4487 | 4.2237 | 33000 | 1.4497 |
| 1.4483 | 4.2877 | 33500 | 1.4497 |
| 1.4492 | 4.3517 | 34000 | 1.4496 |
| 1.4494 | 4.4157 | 34500 | 1.4496 |
| 1.4504 | 4.4797 | 35000 | 1.4496 |
| 1.4501 | 4.5437 | 35500 | 1.4496 |
| 1.4502 | 4.6077 | 36000 | 1.4496 |
| 1.4487 | 4.6717 | 36500 | 1.4496 |
| 1.4492 | 4.7357 | 37000 | 1.4496 |
| 1.4493 | 4.7997 | 37500 | 1.4496 |
| 1.451 | 4.8637 | 38000 | 1.4496 |
| 1.4506 | 4.9277 | 38500 | 1.4496 |
| 1.4484 | 4.9917 | 39000 | 1.4496 |
Framework versions
- Transformers 4.57.1
- Pytorch 2.9.0+cu128
- Datasets 4.5.0
- Tokenizers 0.22.1