africa-female-vc
A Seed-VC voice-conversion model fine-tuned on African speech: convert speech in any language into an African female voice while keeping the words.
- Use it: github.com/AfriSpeech/africa-female-vc - pip-installable library + CLI with 21 built-in voices (not tied to languages - any voice works with speech in any language)
- Try it: AfriSpeech/africa-female-vc-demo
pip install git+https://github.com/AfriSpeech/africa-female-vc
africa-female-vc convert-file talk.wav --voice warm-low -o talk_warm_low.wav
Results
Content preservation after conversion, measured as CER of omniASR-CTC-300M-v2 (sherpa-onnx) on the converted audio against the source transcript. Held-out set: 105 clips (5 per language) from the v2 validation split, each converted into an unseen reference voice of the same language at 50 diffusion steps. The floor is the same ASR on the unconverted source audio - the best a conversion can do.
| CER | above floor | |
|---|---|---|
| Real recordings (floor) | 15.02 | - |
Zero-shot Seed-VC (seed-uvit-whisper-small-wavenet) | 22.57 | +7.55 |
| africa-female-vc | 18.73 | +3.71 |
Per language (5 clips each - treat differences of a point or two as noise):
| Language | Floor | Zero-shot | africa-female-vc |
|---|---|---|---|
| Amharic | 15.2 | 20.8 | 16.7 |
| Chichewa | 5.8 | 11.2 | 11.7 |
| Hausa | 5.7 | 19.9 | 21.2 |
| Igbo | 15.6 | 26.9 | 18.8 |
| Kinyarwanda | 4.8 | 8.2 | 5.8 |
| Kirundi | 4.7 | 10.9 | 9.4 |
| Ndebele | 13.3 | 23.7 | 16.8 |
| Oromo | 15.2 | 22.0 | 12.7 |
| Sepedi | 12.6 | 27.4 | 15.5 |
| Sesotho | 22.4 | 27.4 | 23.0 |
| Setswana | 16.9 | 20.4 | 16.9 |
| Shona | 9.0 | 24.2 | 14.1 |
| Swahili | 2.7 | 7.4 | 4.6 |
| Swati | 15.9 | 21.9 | 16.8 |
| Tigrinya | 21.4 | 42.9 | 26.5 |
| Tsonga | 9.6 | 19.7 | 20.2 |
| Twi | 20.9 | 28.0 | 25.2 |
| Venda | 33.9 | 37.5 | 35.4 |
| Xhosa | 15.8 | 25.4 | 20.6 |
| Yoruba | 48.5 | 37.1 | 50.7 |
| Zulu | 5.3 | 11.3 | 10.7 |
Out-of-domain languages
Measured on AfriSpeech/youversion-african-speech (streamed): 4 clips (4-12 s) per language from different chapters, each converted into a different built-in voice, CER as above (omniASR-CTC-300M-v2 vs the source transcript, 50 diffusion steps).
| Languages | Floor (real audio) | Zero-shot Seed-VC | africa-female-vc | Fine-tuned better in | |
|---|---|---|---|---|---|
| Out-of-domain (never trained on) | 37 | 20.85 | 34.96 (+14.1) | 30.98 (+10.1) | 27/37 |
| Control (training languages, new source) | 4 | 16.27 | 31.73 (+15.5) | 25.37 (+9.1) | 3/4 |
- The fine-tuning transfers to languages the model never saw: the penalty above real speech drops from +14.1 to +10.1 CER points, and the model beats zero-shot Seed-VC in 27 of 37 unseen languages.
- A new language costs about one extra point: the control languages (which the model was trained on) sit at +9.1 on this dataset too, so most of the gap versus the in-domain result comes from the new recording source and voices, not from the language.
- Weakest: Tashelhayt, Mankanya, Fon, Dan and Shilluk (+20 to +42 above floor; several are strongly tonal). Tarifit, Luba-Lulua and Kikuyu are uninformative - the ASR itself scores 55-88% CER on their real audio.
- 4 clips per language: trust the group averages over single-language numbers.
| Language | Floor | Zero-shot | africa-female-vc |
|---|---|---|---|
| Afar | 28.5 | 37.9 | 32.4 |
| Baoulé | 44.2 | 51.0 | 51.4 |
| Bemba (Zambia) | 3.6 | 13.8 | 5.3 |
| Burkina Faso Fulfulde | 9.7 | 28.6 | 21.7 |
| Dan | 6.8 | 37.8 | 28.0 |
| Fon | 23.2 | 56.8 | 45.6 |
| Fulani | 10.6 | 22.6 | 18.8 |
| Ganda | 3.5 | 7.4 | 6.5 |
| Jola-Kasa | 7.7 | 27.4 | 25.6 |
| Kalanga | 14.1 | 20.7 | 17.6 |
| Kalenjin | 23.6 | 31.4 | 28.2 |
| Kikuyu | 54.9 | 62.0 | 58.9 |
| Lingala | 0.9 | 2.1 | 3.4 |
| Lozi | 15.9 | 26.9 | 24.5 |
| Luba-Lulua | 69.2 | 72.2 | 72.4 |
| Makhuwa-Shirima | 14.7 | 31.5 | 13.0 |
| Malgache | 14.7 | 26.6 | 23.1 |
| Mankanya | 6.6 | 38.5 | 40.3 |
| Mbunda | 16.1 | 23.5 | 20.8 |
| Mende | 20.7 | 32.6 | 27.5 |
| Mossi | 10.8 | 19.9 | 18.6 |
| Ngambay | 11.7 | 24.6 | 22.5 |
| Northeastern Dinka | 37.2 | 42.7 | 39.3 |
| Pular | 8.3 | 28.3 | 24.6 |
| Punu (Ipounou) | 16.3 | 25.9 | 26.4 |
| Sango | 6.7 | 22.2 | 24.4 |
| Shilluk | 35.5 | 56.2 | 55.1 |
| Somali | 2.7 | 47.5 | 19.0 |
| Sukuma | 26.5 | 40.1 | 40.3 |
| Tarifit | 88.5 | 91.6 | 87.9 |
| Tashelhayt | 17.8 | 74.9 | 59.9 |
| Tiv | 34.2 | 37.3 | 38.2 |
| Tumbuka | 45.2 | 50.6 | 53.0 |
| Western Niger Fulfulde | 9.1 | 22.7 | 19.5 |
| Wolof (Senegal) | 11.7 | 25.1 | 21.5 |
| Yaka | 13.8 | 17.6 | 18.4 |
| Éwé | 6.1 | 14.9 | 12.7 |
| Amharic (control) | 9.3 | 40.4 | 24.7 |
| Hausa (control) | 37.6 | 45.2 | 42.9 |
| Swahili (control) | 7.5 | 10.2 | 10.5 |
| Yoruba (control) | 10.7 | 31.1 | 23.3 |
Training
- Base:
DiT_seed_v2_uvit_whisper_small_wavenet_bigvgan_pruned.pth(Plachta/Seed-VC), configconfig_dit_mel_seed_uvit_whisper_small_wavenet.yml - Data: AfriSpeech/africa-female-speech-v2-best60 - the cleanest 60 minutes of each language (Demucs-cleaned, UTMOS-ranked), 15,369 clips / 21.0 h, female speakers
- Recipe: batch size 2, up to 30,000 steps (~4 epochs); a checkpoint every 200 steps, each evaluated on a 42-clip held-out set while training ran
- Selected checkpoint: step 25,200 - lowest CER during training, and the best of the final candidates on the independent 105-clip set above
Training curve (42-clip set, 2,000-step windows; zero-shot base 25.42 and floor 17.93 on that set):
| Steps | Mean CER | Mean train loss |
|---|---|---|
| 200-2,000 | 24.11 | 0.596 |
| 2,200-4,000 | 23.33 | 0.594 |
| 4,200-6,000 | 23.15 | 0.593 |
| 6,200-8,000 | 23.19 | 0.594 |
| 8,200-10,000 | 22.88 | 0.589 |
| 10,200-12,000 | 23.00 | 0.590 |
| 12,200-14,000 | 22.80 | 0.590 |
| 14,200-16,000 | 23.21 | 0.592 |
| 16,200-18,000 | 23.07 | 0.586 |
| 18,200-20,000 | 22.95 | 0.588 |
| 20,200-22,000 | 22.78 | 0.588 |
| 22,200-24,000 | 22.81 | 0.586 |
| 24,200-26,000 | 22.83 | 0.586 |
| 26,200-28,000 | 22.68 | 0.588 |
| 28,200-30,000 | 22.27 | 0.586 |
Files
ft_model.pth- fine-tuned DiT weights (no optimizer state)config_dit_mel_seed_uvit_whisper_small_wavenet.yml- Seed-VC config
License
GPL-3.0, following upstream Seed-VC. Training data is CC BY-NC 4.0.