w2v-bert-2.0-hausa_naijavoices_250h
This model is a fine-tuned version of facebook/w2v-bert-2.0 on the CLEAR-GLOBAL/NAIJAVOICES_250H - NA dataset. It achieves the following results on the evaluation set:
- Loss: 0.2333
- Wer: 0.3302
- Cer: 0.1886
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 160
- eval_batch_size: 160
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- total_train_batch_size: 320
- total_eval_batch_size: 320
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100.0
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
|---|---|---|---|---|---|
| 0.3125 | 1.1173 | 1000 | 0.4133 | 0.4180 | 0.2128 |
| 0.2781 | 2.2346 | 2000 | 0.3208 | 0.3702 | 0.1998 |
| 0.2616 | 3.3520 | 3000 | 0.2858 | 0.3534 | 0.1950 |
| 0.2068 | 4.4693 | 4000 | 0.2772 | 0.3420 | 0.1923 |
| 0.3346 | 5.5866 | 5000 | 0.2620 | 0.3419 | 0.1915 |
| 0.2176 | 6.7039 | 6000 | 0.2535 | 0.3410 | 0.1916 |
| 0.1969 | 7.8212 | 7000 | 0.2466 | 0.3380 | 0.1909 |
| 0.1992 | 8.9385 | 8000 | 0.2467 | 0.3322 | 0.1896 |
| 0.197 | 10.0559 | 9000 | 0.2489 | 0.3322 | 0.1893 |
| 0.1832 | 11.1732 | 10000 | 0.2533 | 0.3385 | 0.1909 |
| 0.1859 | 12.2905 | 11000 | 0.2410 | 0.3334 | 0.1898 |
| 0.1751 | 13.4078 | 12000 | 0.2419 | 0.3307 | 0.1886 |
| 0.1569 | 14.5251 | 13000 | 0.2418 | 0.3321 | 0.1895 |
| 0.2345 | 15.6425 | 14000 | 0.2480 | 0.3352 | 0.1898 |
| 0.1769 | 16.7598 | 15000 | 0.2365 | 0.3280 | 0.1889 |
| 0.1634 | 17.8771 | 16000 | 0.2401 | 0.3345 | 0.1895 |
| 0.1647 | 18.9944 | 17000 | 0.2329 | 0.3303 | 0.1888 |
| 0.1937 | 20.1117 | 18000 | 0.2485 | 0.3273 | 0.1881 |
| 0.1671 | 21.2291 | 19000 | 0.2568 | 0.3327 | 0.1891 |
| 0.1229 | 22.3464 | 20000 | 0.2460 | 0.3304 | 0.1887 |
| 0.1252 | 23.4637 | 21000 | 0.2437 | 0.3337 | 0.1889 |
| 0.1186 | 24.5810 | 22000 | 0.2544 | 0.3309 | 0.1892 |
Framework versions
- Transformers 4.48.1
- Pytorch 2.6.0+cu124
- Datasets 3.5.0
- Tokenizers 0.21.1