IndicF5 fine-tuned on a single Bangladeshi Bengali speaker
ai4bharat/IndicF5 fine-tuned on 15.89 h of one
female Bangladeshi Bengali speaker. Same architecture, same tokeniser, same frozen Vocos
vocoder — only the DiT weights differ, so this file is a drop-in replacement for the base
model's model.safetensors.
337.1M parameters · 24 kHz · 100-bin mel · flow matching (32 NFE default).
Evaluation
100 held-out sentences (90 single utterances + 10 passages) from speaking sessions the model
never saw, scored against a 20-clip reference bank taken from those same held-out sessions but
disjoint from the scored items. CER uses bengaliAI/tugstugi_bengaliai-asr_whisper-medium;
SECS is speechbrain/spkrec-ecapa-voxceleb cosine similarity.
Usage
from transformers import AutoModel
import soundfile as sf
model = AutoModel.from_pretrained("ehzawad/indicf5-bangla-tts", trust_remote_code=True)
audio = model(
"আমাদের দেশের নদীগুলো এখন অনেক শুকিয়ে গেছে।",
ref_audio_path="reference.wav", # a few seconds of the voice to speak in
ref_text="reference transcript", # what that clip says
)
sf.write("out.wav", audio, samplerate=24000)
Voice cloning is zero-shot from ref_audio_path, exactly as in the base model. Fine-tuning
shifted the prior toward one speaker; it did not remove the reference conditioning.
Two things that will silently give you the wrong model
- A pip-installed
f5_ttsshadows the vendored one.model.pydoessys.path.append(current_dir), which appends, so an installedf5-ttspackage wins the import. The vendoredf5_tts/in this repo is an older F5-TTS whose DiT defaults differ (text_mask_padding=False,pe_attn_head=1). Build with upstream defaults and the weights still load with zero missing keys and still produce plausible audio — it is simply a different network. Either uninstallf5-tts, or construct the DiT yourself with those two values set. use_epssmust beFalse. Upstream F5-TTS added Empirically Pruned Step Sampling and turns it on by default. The vendored sampler has no such notion and uses a uniformlinspaceschedule. Leaving EPSS on changes every sample.
The training repo ships a test that asserts bit-exactness against the vendored implementation and fails if either default drifts.
Training
| data | 12,798 utterances / 15.89 h, one speaker, session-disjoint splits |
| learning rate | 3e-5, 100-update warm-up, linear decay |
| updates | 2545 (3 epochs) |
| batch | dynamic, 8192 frames × accum 2 = 16384 frames/update |
| optimiser | AdamW 8-bit, gradient clipping 1.0 |
| precision | bf16 autocast over fp32 parameters |
| hardware | one RTX A5000, 6.10 GiB peak, ≈1.9 h |
Validation loss 0.75822 → 0.65251 (−13.9 %). The shipped weights are the raw weights at update 2500, not the EMA copy: EMA had the marginally better validation loss but a worse worst-case CER on the dev set (0.121 against 0.064).
One naming caveat: the tensors are stored under ema_model._orig_mod.* keys, because that is the layout ai4bharat/IndicF5 uses and keeping it is what makes this file drop-in. The keys say ema_model; the values are the raw weights.
Limitations
- One voice, one accent, one recording chain. Bangladeshi Bengali, read speech, studio conditions. It has not been evaluated on West Bengal Bengali, on conversational speech, or on noisy references.
- No numerals. Lines containing Bengali or ASCII digits were excluded from training, because the corpus gives no spoken form for them. Verbalise numbers before synthesis.
- No emotion or style control. The target was natural neutral speech.
- Machine metrics only. CER and SECS are proxies. Nobody has run a listening study.
Terms
Inherited from the base model: only clone voices you have explicit permission to clone. Unauthorised voice cloning is prohibited, and misuse is the responsibility of the user.
Credits
ai4bharat/IndicF5— base model, MIT.model.pyand the vendoredf5_tts/in this repo are theirs, redistributed unchanged.- SWivid/F5-TTS — architecture, MIT
- Vocos — vocoder, MIT
- BnTTS — the speaker and the corpus