wav2vec-vm-finetune
This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m for voicemail detection. It is trained on a dataset of call recordings to distinguish between voicemail greetings and live human responses.
Model description
This model builds on wav2vec2-xls-r-300m, a self-supervised speech model trained on large-scale multilingual data. We fine-tuned it on the first two seconds of a call.
Intended uses & limitations
-
Automated voicemail detection in AI-powered call assistants.
-
Filtering voicemail responses in customer service and sales call automation.
-
Only trianed on the English language.
-
Assumes the voicemail track is isolated and contains no audio from the caller.
-
Designed for the first two seconds of audio when calling a voicemail.
Training and evaluation data
The model was trained on a proprietary dataset of call recordings, labeled as:
- Live human responses
- Voicemail greetings
The dataset includes diverse voicemail recordings across multiple types to improve generalization.
Evaluation metrics
The model achieved:
- 98% accuracy on voicemail detection.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP
Framework versions
- Transformers 4.48.2
- Pytorch 2.5.1+cu124
- Datasets 1.18.3
- Tokenizers 0.21.0