Matcha-TTS Nepali (Chatterbox, single speaker, English+Nepali mixed)
Fine-tuned Matcha-TTS acoustic model for Nepali (with mixed English words in Latin script) built on the chatterbox-nepali dataset, warm-started from the pretrained English (LJSpeech) base checkpoint.
Model summary
| Property | Value |
|---|---|
| Architecture | Matcha-TTS (CFM decoder, RoPE encoder) |
| Vocab | union (n_vocab=264): English IPA (178) + real_nepali G2P phones preserved at 0..177 so the English base loads 100% |
| G2P | real_nepali profile real_nepali_v0.2 (Devanagari) + espeak-ng en-us IPA (Latin) |
| Speakers | 1 (single speaker) |
| Sample rate | 22,050 Hz |
| Mel | n_feats=80, n_fft=1024, hop=256, f_max=8000; mel_mean=-5.980818, mel_std=2.582703 |
| Warm start | pretrained/matcha_tts_ljspeech_english.ckpt via extend_vocab.py |
| Optimisation | Adam lr=1e-4 constant, batch 32 x grad-accum 2 (eff. 64), 16-mixed precision, grad-clip 5.0 |
| Training | 150-epoch target, early-stopped (see Re-generation) |
Files
| Path | Description |
|---|---|
checkpoints/matcha_nepali_final.ckpt | Final acoustic model (epoch 999 of the original run, picked by listening) |
configs/nepali_real_fresh.yaml | Experiment config (hydra) used for training |
configs/nepali_real.yaml | Data config (filelists, batch, mel stats) |
configs/matcha.yaml | Model config (architecture, optimizer, scheduler) |
scripts/synthesize.py | Inference script (union symbols, denoiser, mel-stats matching) |
scripts/extend_vocab.py | Extends the English base ckpt to the 264 union vocab |
Usage
.venv/bin/python scripts/synthesize.py \
--ckpt checkpoints/matcha_nepali_final.ckpt \
--file texts.txt --out outputs --symbols union \
--steps 25 --denoiser_strength 0.02
One utterance per line in texts.txt. Mixed Devanagari + Latin works in the
same sentence, e.g. नमस्ते, welcome to our bank.
Recommended synthesis settings (cleanest output):
--steps 25(more ODE steps than the 10 default = less noise)--denoiser_strength 0.02(vs default 0.00025: removes high-frequency hiss)
Vocoder: universal HiFi-GAN hifigan_univ_v1 (g_02500000).
Regeneration
-
Prepare the dataset (split + phonemize with real_nepali G2P), loudness-normalize to
data/wavs_norm/, set mel stats in the data config. -
Extend the English base to the union vocab:
.venv/bin/python scripts/extend_vocab.py \ --src pretrained/matcha_tts_ljspeech_english.ckpt \ --out checkpoints/matcha_tts_english_plus_nepali.ckpt -
Train:
bash scripts/train.sh fresh
10-15k steps on a 1.5k-utterance single-speaker set is enough; longer runs overfit (val loss rises again ~epoch 126) which adds noise and degrades the English-prosody carried from the base model.
Notes
- The English base checkpoint is MIT (Shivam Mehta et al., ICASSP 2024); training data comes from the chatterbox-nepali dataset (CC-BY-SA-4.0).
- The real_nepali G2P frontend (CC-BY-SA-4.0, Ampixa Labs) is required at train/inference time for Devanagari text.
- This upload (weights + docs) is CC-BY-4.0.