wav2vec2-xls-r-1b-distant-from-faroese-30-epochs_20250120_v5
This model is a fine-tuned version of davidilag/wav2vec2-xls-r-1b-distant-from-faroese-251h-30-epochs_20250120_v2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.3469
- Wer: 30.3068
- Cer: 5.9533
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Lan=Latvian
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 3000
- num_epochs: 30
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
|---|---|---|---|---|---|
| 0.8208 | 1.2563 | 1000 | 0.4074 | 47.8193 | 11.7942 |
| 0.3476 | 2.5126 | 2000 | 0.1536 | 28.0904 | 6.2627 |
| 0.3526 | 3.7688 | 3000 | 0.1304 | 25.9786 | 5.5754 |
| 0.2716 | 5.0251 | 4000 | 0.1318 | 25.2509 | 5.3006 |
| 0.2212 | 6.2814 | 5000 | 0.1175 | 25.2985 | 5.2105 |
| 0.2388 | 7.5377 | 6000 | 0.1312 | 26.5018 | 5.3714 |
| 0.6469 | 8.7940 | 7000 | 0.4201 | 29.6599 | 5.8394 |
| 0.6762 | 10.0503 | 8000 | 0.4197 | 28.3092 | 5.8971 |
| 0.5835 | 11.3065 | 9000 | 0.3591 | 28.2093 | 5.7462 |
| 0.6136 | 12.5628 | 10000 | 0.3550 | 28.1760 | 5.6700 |
| 0.6017 | 13.8191 | 11000 | 0.3506 | 29.7646 | 5.8825 |
| 0.5743 | 15.0754 | 12000 | 0.3480 | 30.2640 | 5.9464 |
| 0.6166 | 16.3317 | 13000 | 0.3469 | 30.3020 | 5.9502 |
| 0.5857 | 17.5879 | 14000 | 0.3469 | 30.3020 | 5.9525 |
| 0.5936 | 18.8442 | 15000 | 0.3469 | 30.2878 | 5.9502 |
| 0.5648 | 20.1005 | 16000 | 0.3469 | 30.2925 | 5.9518 |
| 0.5515 | 21.3568 | 17000 | 0.3469 | 30.3020 | 5.9510 |
| 0.5667 | 22.6131 | 18000 | 0.3469 | 30.3020 | 5.9510 |
| 0.567 | 23.8693 | 19000 | 0.3469 | 30.3115 | 5.9518 |
| 0.6011 | 25.1256 | 20000 | 0.3469 | 30.3068 | 5.9525 |
| 0.5903 | 26.3819 | 21000 | 0.3469 | 30.2973 | 5.9510 |
| 0.5791 | 27.6382 | 22000 | 0.3469 | 30.3068 | 5.9525 |
| 0.6089 | 28.8945 | 23000 | 0.3469 | 30.3068 | 5.9533 |
Framework versions
- Transformers 4.48.0
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0