w2vbert-shona-waxal-punct-v2
This model is a fine-tuned version of sulaimank/w2vbert-shona-waxal on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0337
- Wer Keep: 0.0995
- Cer Keep: 0.0134
- Zindi Keep: 0.9435
- Wer Strip: 0.0547
- Cer Strip: 0.0124
- Zindi Strip: 0.9664
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 0.1
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer Keep | Cer Keep | Zindi Keep | Wer Strip | Cer Strip | Zindi Strip |
|---|---|---|---|---|---|---|---|---|---|
| 0.1990 | 0.1329 | 200 | 0.0584 | 0.1276 | 0.0185 | 0.9269 | 0.0768 | 0.0169 | 0.9531 |
| 0.1195 | 0.2659 | 400 | 0.0389 | 0.1116 | 0.0151 | 0.9367 | 0.0635 | 0.0140 | 0.9612 |
| 0.1028 | 0.3988 | 600 | 0.0370 | 0.1071 | 0.0145 | 0.9392 | 0.0609 | 0.0134 | 0.9628 |
| 0.0941 | 0.5317 | 800 | 0.0355 | 0.1086 | 0.0145 | 0.9385 | 0.0633 | 0.0135 | 0.9616 |
| 0.0945 | 0.6647 | 1000 | 0.0348 | 0.1055 | 0.0139 | 0.9403 | 0.0616 | 0.0133 | 0.9625 |
| 0.0985 | 0.7976 | 1200 | 0.0335 | 0.1062 | 0.0141 | 0.9398 | 0.0625 | 0.0135 | 0.9620 |
| 0.0976 | 0.9305 | 1400 | 0.0325 | 0.1056 | 0.0140 | 0.9402 | 0.0612 | 0.0132 | 0.9628 |
| 0.0677 | 1.0631 | 1600 | 0.0341 | 0.1053 | 0.0144 | 0.9401 | 0.0607 | 0.0132 | 0.9631 |
| 0.0752 | 1.1961 | 1800 | 0.0322 | 0.1039 | 0.0142 | 0.9409 | 0.0591 | 0.0130 | 0.9639 |
| 0.0725 | 1.3290 | 2000 | 0.0327 | 0.1040 | 0.0143 | 0.9409 | 0.0599 | 0.0130 | 0.9636 |
| 0.0716 | 1.4619 | 2200 | 0.0341 | 0.1025 | 0.0135 | 0.9420 | 0.0581 | 0.0128 | 0.9645 |
| 0.0667 | 1.5949 | 2400 | 0.0326 | 0.1019 | 0.0138 | 0.9422 | 0.0585 | 0.0129 | 0.9643 |
| 0.0678 | 1.7278 | 2600 | 0.0315 | 0.1017 | 0.0143 | 0.9420 | 0.0569 | 0.0127 | 0.9652 |
| 0.0632 | 1.8608 | 2800 | 0.0335 | 0.1023 | 0.0139 | 0.9419 | 0.0581 | 0.0129 | 0.9645 |
| 0.0629 | 1.9937 | 3000 | 0.0324 | 0.1015 | 0.0138 | 0.9423 | 0.0565 | 0.0127 | 0.9654 |
| 0.0411 | 2.1263 | 3200 | 0.0343 | 0.1007 | 0.0135 | 0.9429 | 0.0561 | 0.0125 | 0.9657 |
| 0.0444 | 2.2592 | 3400 | 0.0335 | 0.0998 | 0.0136 | 0.9433 | 0.0560 | 0.0126 | 0.9657 |
| 0.0359 | 2.3922 | 3600 | 0.0348 | 0.1008 | 0.0138 | 0.9427 | 0.0557 | 0.0125 | 0.9659 |
| 0.0410 | 2.5251 | 3800 | 0.0342 | 0.0989 | 0.0133 | 0.9439 | 0.0548 | 0.0124 | 0.9664 |
| 0.0383 | 2.6580 | 4000 | 0.0354 | 0.0998 | 0.0135 | 0.9433 | 0.0552 | 0.0124 | 0.9662 |
| 0.0371 | 2.7910 | 4200 | 0.0347 | 0.0999 | 0.0136 | 0.9433 | 0.0551 | 0.0124 | 0.9662 |
| 0.0366 | 2.9239 | 4400 | 0.0344 | 0.0990 | 0.0134 | 0.9438 | 0.0547 | 0.0124 | 0.9665 |
| 0.0466 | 3.0 | 4515 | 0.0337 | 0.0995 | 0.0134 | 0.9435 | 0.0547 | 0.0124 | 0.9664 |
Framework versions
- Transformers 5.14.1
- Pytorch 2.13.0+cu130
- Datasets 3.6.0
- Tokenizers 0.22.2