VoiceClone — F5-TTS fine-tune
Zero-shot voice cloning: give it a 6–15 s reference clip and arbitrary text, and it
synthesizes that text in the reference speaker's voice. Fine-tuned from
SWivid/F5-TTS
(F5TTS_v1_Base, ~336M params, 24 kHz, Vocos vocoder).
Non-commercial. The base weights inherit CC-BY-NC-4.0 from the Emilia corpus, and that restriction flows through to this fine-tune. See Licensing below.
Current checkpoint
| Training step | 12000 |
| Precision | bf16, full fine-tune (no LoRA) |
| Sample rate | 24000 Hz |
Evaluation
Measured on held-out speakers never seen during training, with sentences never seen during training — so these numbers reflect zero-shot generalisation, not memorisation.
| Metric | Value | What it means |
|---|---|---|
| SECS ↑ | 0.7937 (+0.0067 ✅ vs base) | Speaker-encoder cosine similarity (ECAPA-TDNN). The primary "is cloning working" metric. |
| WER ↓ | 1.70% (+0.0000 ⚠️ vs base) | Whisper transcription vs input text. Catches intelligibility collapse. |
| CER ↓ | 1.46% | Character error rate. |
| UTMOS ↑ | 4.226 | Predicted mean opinion score (naturalness). |
| RTF ↓ | 0.291 | Real-time factor on an RTX 5070 Ti. |
MCD is tracked internally but deliberately not published here: it is computed from
MFCC rather than WORLD/SPTK mel-cepstrum, so it is valid for tracking this model's own
progress but is not comparable to MCD figures in the literature.
Samples
samples/000_libritts_r_2053.wavsamples/001_libritts_r_2053.wavsamples/002_libritts_r_2053.wavsamples/003_libritts_r_2053.wavsamples/004_libritts_r_2053.wavsamples/005_vctk_p283.wavsamples/006_vctk_p283.wavsamples/007_vctk_p283.wav
Usage
from f5_tts.api import F5TTS
tts = F5TTS(model="F5TTS_v1_Base", ckpt_file="model.safetensors")
wav, sr, _ = tts.infer(
ref_file="reference.wav",
ref_text="Transcript of the reference clip.",
gen_text="Text you want spoken in that voice.",
file_wave="out.wav",
)
Reference clips should be 6–15 s, clean, and single-speaker. Quality of the reference dominates output quality far more than the length does.
Training data
| Source | HF id | License | Attribution |
|---|---|---|---|
| libritts_r | mythicinfinity/libritts_r | CC BY 4.0 | LibriTTS-R (Koizumi et al., 2023), derived from LibriTTS / LibriVox. |
| vctk | sanchit-gandhi/vctk | CC BY 4.0 (upstream CSTR VCTK; mirror declares none) | CSTR VCTK Corpus, University of Edinburgh (CC BY 4.0). Accessed via the sanchit-gandhi/vctk parquet mirror. |
Audio was resampled to 24000 Hz mono, loudness-normalised to -23.0 LUFS (EBU R128), VAD-trimmed, and filtered on duration (1.0–15.0 s), estimated SNR (≥ 12.0 dB), clipping, and transcript plausibility.
Licensing
- This model:
cc-by-nc-4.0— non-commercial use only. - Base weights (
SWivid/F5-TTS) are CC-BY-NC-4.0 because the Emilia training corpus is CC-BY-NC. That restriction is viral onto any fine-tune, including this one. - F5-TTS code is MIT.
- LibriTTS-R and VCTK are CC BY 4.0 and require attribution, given above.
For a commercially usable lineage you would need to retrain from an Apache-2.0 base such
as mrfakename/OpenF5-TTS-Base (identical architecture) — fine-tuning this checkpoint
cannot remove the NC restriction.
Ethical use
Voice cloning can be used to impersonate people. Only clone voices you have explicit permission to clone. Do not use this model for fraud, harassment, defamation, or to fabricate statements by real people.
Auto-generated by hf_push.py at 2026-07-27 04:31:46.