w2vbert-luganda-waxal-punct
This model is a fine-tuned version of sulaimank/w2vbert-luganda-waxal-stage2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.1886
- Wer Keep: 0.1881
- Cer Keep: 0.0361
- Zindi Keep: 0.8879
- Wer Strip: 0.1055
- Cer Strip: 0.0230
- Zindi Strip: 0.9358
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 0.1
- num_epochs: 40
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer Keep | Cer Keep | Zindi Keep | Wer Strip | Cer Strip | Zindi Strip |
|---|---|---|---|---|---|---|---|---|---|
| 17.4208 | 0.2950 | 50 | 7.9497 | 1.0 | 1.5245 | -0.2622 | 1.0587 | 1.4766 | -0.2677 |
| 12.4278 | 0.5900 | 100 | 5.5352 | 1.0 | 0.9198 | 0.0401 | 1.0 | 0.9142 | 0.0429 |
| 7.8258 | 0.8850 | 150 | 3.5134 | 0.9996 | 0.8322 | 0.0841 | 0.9996 | 0.8314 | 0.0845 |
| 6.1205 | 1.1770 | 200 | 3.0291 | 1.0 | 0.9893 | 0.0054 | 1.0 | 0.9898 | 0.0051 |
| 5.9425 | 1.4720 | 250 | 2.9982 | 1.0 | 0.9944 | 0.0028 | 0.9998 | 0.9951 | 0.0026 |
| 5.7308 | 1.7670 | 300 | 2.7803 | 0.9998 | 0.9750 | 0.0126 | 0.9998 | 0.9805 | 0.0098 |
| 1.8717 | 2.0590 | 350 | 0.4798 | 0.5119 | 0.1298 | 0.6791 | 0.4785 | 0.1158 | 0.7029 |
| 0.4323 | 2.3540 | 400 | 0.2118 | 0.2535 | 0.0487 | 0.8489 | 0.1633 | 0.0331 | 0.9018 |
| 0.3656 | 2.6490 | 450 | 0.1767 | 0.2331 | 0.0449 | 0.8610 | 0.1379 | 0.0292 | 0.9164 |
| 0.2574 | 2.9440 | 500 | 0.1602 | 0.2193 | 0.0417 | 0.8695 | 0.1223 | 0.0257 | 0.9260 |
| 0.2359 | 3.2360 | 550 | 0.1578 | 0.2133 | 0.0408 | 0.8729 | 0.1154 | 0.0252 | 0.9297 |
| 0.2406 | 3.5310 | 600 | 0.1469 | 0.2039 | 0.0395 | 0.8783 | 0.1124 | 0.0245 | 0.9316 |
| 0.2034 | 3.8260 | 650 | 0.1493 | 0.2013 | 0.0386 | 0.8801 | 0.1130 | 0.0245 | 0.9312 |
| 0.2034 | 4.1180 | 700 | 0.1420 | 0.1939 | 0.0376 | 0.8842 | 0.1068 | 0.0239 | 0.9346 |
| 0.2003 | 4.4130 | 750 | 0.1496 | 0.1957 | 0.0377 | 0.8833 | 0.1103 | 0.0242 | 0.9328 |
| 0.1792 | 4.7080 | 800 | 0.1423 | 0.1952 | 0.0379 | 0.8834 | 0.1090 | 0.0242 | 0.9334 |
| 0.1571 | 5.0 | 850 | 0.1419 | 0.1977 | 0.0378 | 0.8823 | 0.1087 | 0.0236 | 0.9338 |
| 0.1336 | 5.2950 | 900 | 0.1408 | 0.1976 | 0.0376 | 0.8824 | 0.1095 | 0.0236 | 0.9334 |
| 0.1750 | 5.5900 | 950 | 0.1508 | 0.1866 | 0.0363 | 0.8886 | 0.1038 | 0.0232 | 0.9365 |
| 0.1477 | 5.8850 | 1000 | 0.1433 | 0.1844 | 0.0359 | 0.8898 | 0.1029 | 0.0229 | 0.9371 |
| 0.1089 | 6.1770 | 1050 | 0.1486 | 0.1844 | 0.0359 | 0.8899 | 0.1016 | 0.0227 | 0.9379 |
| 0.1218 | 6.4720 | 1100 | 0.1546 | 0.1869 | 0.0358 | 0.8887 | 0.1048 | 0.0228 | 0.9362 |
| 0.1414 | 6.7670 | 1150 | 0.1592 | 0.1846 | 0.0358 | 0.8898 | 0.1038 | 0.0230 | 0.9366 |
| 0.1083 | 7.0590 | 1200 | 0.1567 | 0.1857 | 0.0357 | 0.8893 | 0.1046 | 0.0227 | 0.9363 |
| 0.0972 | 7.3540 | 1250 | 0.1593 | 0.1847 | 0.0360 | 0.8897 | 0.1045 | 0.0232 | 0.9362 |
| 0.1106 | 7.6490 | 1300 | 0.1616 | 0.1882 | 0.0369 | 0.8874 | 0.1071 | 0.0241 | 0.9344 |
| 0.1129 | 7.9440 | 1350 | 0.1658 | 0.1871 | 0.0368 | 0.8881 | 0.1069 | 0.0240 | 0.9346 |
| 0.0939 | 8.2360 | 1400 | 0.1698 | 0.1842 | 0.0363 | 0.8898 | 0.1044 | 0.0234 | 0.9361 |
| 0.0942 | 8.5310 | 1450 | 0.1513 | 0.1855 | 0.0366 | 0.8890 | 0.1043 | 0.0237 | 0.9360 |
| 0.0891 | 8.8260 | 1500 | 0.1586 | 0.1860 | 0.0360 | 0.8890 | 0.1049 | 0.0231 | 0.9360 |
| 0.0685 | 9.1180 | 1550 | 0.1718 | 0.1897 | 0.0367 | 0.8868 | 0.1043 | 0.0230 | 0.9363 |
| 0.0788 | 9.4130 | 1600 | 0.1805 | 0.1883 | 0.0362 | 0.8877 | 0.1046 | 0.0231 | 0.9362 |
| 0.0788 | 9.7080 | 1650 | 0.1740 | 0.1881 | 0.0367 | 0.8876 | 0.1067 | 0.0236 | 0.9348 |
| 0.0665 | 10.0 | 1700 | 0.1740 | 0.1879 | 0.0359 | 0.8881 | 0.1057 | 0.0229 | 0.9357 |
| 0.0711 | 10.2950 | 1750 | 0.1867 | 0.1872 | 0.0365 | 0.8881 | 0.1052 | 0.0235 | 0.9356 |
| 0.0756 | 10.5900 | 1800 | 0.1886 | 0.1881 | 0.0361 | 0.8879 | 0.1055 | 0.0230 | 0.9358 |
Framework versions
- Transformers 5.12.1
- Pytorch 2.6.0+cu124
- Datasets 3.6.0
- Tokenizers 0.22.2