Whisper-Small Fine-Tuned for Santali Speech-to-Sanlish Transliteration

This repository contains a fine-tuned Whisper-small model for Santali speech-to-Sanlish transliteration.

The model was fine-tuned on a small Santali speech dataset to convert spoken Santali into Roman-script Santali (Sanlish). The model is intended for research on low-resource speech recognition and Santali language technology.

Model Details

• Base model: openai/whisper-small
• Task: Santali Speech → Sanlish Text
• Model type: Transformer-based encoder-decoder ASR
• Language: Santali
• Output: Sanlish (Romanized Santali)
• Learning rate: 2e-5
• Maximum planned epochs: 25
• Completed epochs: 14
• Training samples: 1,489
• Validation samples: 192
• Test samples: 194

Dataset

The model was fine-tuned using a Santali speech dataset containing paired audio and Sanlish transcriptions.

The dataset was divided into:

The dataset is designed for developing speech technology for Santali, a low-resource language.

Training

The model was fine-tuned from the pretrained Whisper-small model.

Training Configuration

Training started from the pretrained Whisper-small model rather than resuming from an existing checkpoint.

Training Progress

The following validation results were obtained during training:

The lowest validation WER was observed at epoch 9 (38.5469%).

The lowest validation CER was also observed at epoch 9 (8.5214%).

Test Results

Evaluation on the held-out test set produced:

The test results were saved separately as test_scores.json.

{
  "hub_model_id": "thunderboltc/whisper_sanlish_mnx_lr2e5",
  "learning_rate": 2e-05,
  "num_train_epochs": 25,
  "test_wer": 37.5,
  "test_cer": 8.273092369477911,
  "n_train": 1489,
  "n_val": 192,
  "n_test": 194
}