ssc-top-mms-model-mix-adapt-max
This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.7289
- Cer: 0.1445
- Wer: 0.5792
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 8
- eval_batch_size: 6
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 5
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Cer | Wer |
|---|---|---|---|---|---|
| 1.3243 | 0.3524 | 200 | 0.8187 | 0.1612 | 0.6843 |
| 1.2349 | 0.7048 | 400 | 0.7650 | 0.1595 | 0.6707 |
| 1.2417 | 1.0564 | 600 | 0.7351 | 0.1513 | 0.6322 |
| 1.2134 | 1.4088 | 800 | 0.7760 | 0.1521 | 0.6384 |
| 1.1341 | 1.7612 | 1000 | 0.7522 | 0.1478 | 0.5995 |
| 1.1513 | 2.1128 | 1200 | 0.7333 | 0.1489 | 0.6002 |
| 1.0074 | 2.4652 | 1400 | 0.7404 | 0.1495 | 0.6197 |
| 1.0128 | 2.8176 | 1600 | 0.7439 | 0.1515 | 0.6232 |
| 1.0346 | 3.1692 | 1800 | 0.7357 | 0.1478 | 0.6088 |
| 1.005 | 3.5216 | 2000 | 0.7420 | 0.1483 | 0.6065 |
| 0.9878 | 3.8740 | 2200 | 0.7508 | 0.1468 | 0.6041 |
| 0.9541 | 4.2256 | 2400 | 0.7342 | 0.1470 | 0.6014 |
| 0.9563 | 4.5780 | 2600 | 0.7314 | 0.1449 | 0.5812 |
| 0.9478 | 4.9304 | 2800 | 0.7289 | 0.1445 | 0.5792 |
Framework versions
- Transformers 4.57.2
- Pytorch 2.9.1+cu128
- Datasets 3.6.0
- Tokenizers 0.22.0