Whisper Medium CV17 Es 500 steps trained with our def5 on mmarron14/fixie_removed_columns_16kHz_lengths that is without processing of text
This model is a fine-tuned version of openai/whisper-medium trained on our dataset mmarron14/fixie_removed_columns_16kHz_lengths, that is, our resampled to 16kHz version of fixie-ai/common_voice_17_0 dataset without text processing. It achieves the following results on the evaluation set:
- Loss: 0.1753
- Wer Ortho: 10.8189
- Wer: 6.1590
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- training_steps: 500
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer Ortho | Wer |
|---|---|---|---|---|---|
| 0.2133 | 0.2 | 100 | 0.1960 | 11.7515 | 7.0215 |
| 0.1951 | 0.4 | 200 | 0.1895 | 11.3715 | 6.5620 |
| 0.2039 | 0.6 | 300 | 0.1831 | 11.2734 | 6.5289 |
| 0.1836 | 0.8 | 400 | 0.1773 | 10.8953 | 6.2499 |
| 0.1752 | 1.0 | 500 | 0.1753 | 10.8189 | 6.1590 |
Framework versions
- Transformers 4.53.2
- Pytorch 2.11.0+cu128
- Datasets 4.0.0
- Tokenizers 0.21.4
Dataset Sources
- Original dataset repository: fixie-ai/common_voice_17_0
- Our version of the dataset's Repository : mmarron14/fixie_removed_columns_16kHz_lengths
Citation
-
Fixie-ai (2024). Common Voice (Version 17.0). [Dataset]. Hugging Face. (https://huggingface.co/datasets/fixie-ai/common_voice_17_0)
-
Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F. M., & Weber, G. (2020). Common voice: A massively-multilingual speech corpus. LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings, 4218-4222.
-
Radford, Alec & Kim, Jong & Xu, Tao & Brockman, Greg & McLeavey, Christine & Sutskever, Ilya. (2022). Robust Speech Recognition via Large-Scale Weak Supervision. 10.48550/arXiv.2212.04356.