Vocal intensity conversion with continuous SPL control via adversarial training
Quentin Le Tellier, Albert Rilliard, Olivier Perrotin, Marc Evrard Submitted to ICASSP 2027.
These are the trained converters of the paper. A converter renders a recorded utterance at a target vocal intensity, given in dB SPL at 1 m, while preserving its content and speaker identity. It transforms the features of a frozen speech encoder, and a pretrained vocoder resynthesizes the speech.
Models
| folder | encoder/decoder pair | parameters |
|---|---|---|
converter-wavlm | WavLM-Large layer 6 + kNN-VC HiFi-GAN | 2.66 M |
Each folder holds the converter weights (model.safetensors) and the
configuration needed to rebuild it (config.yaml). The encoder and vocoder are
not included: the code downloads them from their original sources.
Usage
git clone https://github.com/vers-project/vocal-intensity-conversion.git
cd vocal-intensity-conversion
uv run --extra cpu --extra hub scripts/convert.py \
--model vers-project/vocal-intensity-conversion \
--subfolder converter-wavlm \
--input speech.wav \
--target-db 70.0 \
--output speech_70dB.wav
The first run downloads WavLM-Large and the kNN-VC HiFi-GAN (about 1.3 GB)
into ~/.cache/vic.
Training
The converter is a Transformer encoder (six layers, width 128, four heads), trained as a conditional cycle-consistent GAN without parallel data, on the training split of AVID, the Aalto Vocal Intensity Database (CC BY 4.0). It was trained for 11 400 iterations, as reported in the paper.
Limitations
The converter was trained on AVID only. Target intensities outside the range seen in training, 42.8 to 76.1 dB SPL, were not evaluated.
Third-party components
- WavLM-Large (Chen et al., 2022) and the HiFi-GAN vocoder of kNN-VC (Baas et al., 2023), both from https://github.com/bshall/knn-vc (MIT).
Citation
@misc{letellier2027vocal,
title = {Vocal Intensity Conversion with Continuous {SPL} Control via Adversarial Training},
author = {Le Tellier, Quentin and Rilliard, Albert and Perrotin, Olivier and Evrard, Marc},
year = {2026},
note = {Submitted to ICASSP 2027}
}
Licence
MIT.