Nepali Parler-TTS -- Finetuned
Nepali finetuned version of Indic Parler-TTS.
Training config
| Parameter | Value |
|---|
| Base model | ai4bharat/indic-parler-tts-pretrained |
| Resumed from | N/A |
| GPU | DGX |
| Epochs | 2 |
| Batch size | 32 |
| Grad accum steps | 2 |
| Effective batch size | 64 |
| Learning rate | 3e-06 |
| LR schedule | cosine with warmup |
| Warmup steps | 62.8 |
| Total steps | 628 |
| Max audio tokens | 550 |
| Precision | bfloat16 |
| Grad clip norm | 1.0 |
Results
| Metric | Value |
|---|
| Best val loss | 5.4939 |
| Steps trained | 628 |
| Training time | 135.3 min |
Usage
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import torch, soundfile as sf
model_id = "milanakdj/indic-parler-tts-nepali-finetuned-dgx-v2.4-cosine-oov"
device = "cuda" if torch.cuda.is_available() else "cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained(model_id).to(device)
prompt_tokenizer = AutoTokenizer.from_pretrained(model_id)
desc_tokenizer = AutoTokenizer.from_pretrained(model.config.text_encoder._name_or_path)
prompt = "नमस्ते, तपाईंलाई कस्तो छ?"
description = "Shristi speaks clearly in Nepali at a steady pace. Very high quality audio."
desc_enc = desc_tokenizer(description, return_tensors="pt").to(device)
prompt_enc = prompt_tokenizer(prompt, return_tensors="pt").to(device)
with torch.inference_mode():
gen = model.generate(
input_ids=desc_enc.input_ids,
attention_mask=desc_enc.attention_mask,
prompt_input_ids=prompt_enc.input_ids,
prompt_attention_mask=prompt_enc.attention_mask,
)
sf.write("out.wav", gen.cpu().numpy().squeeze(), model.config.sampling_rate)