Chatterbox-MTL Central-Asian (Tajik · Uzbek · Kazakh · Kyrgyz)
A single text-to-speech model that speaks four Central-Asian languages — Tajik,
Uzbek, Kazakh and Kyrgyz — with the target language selected at inference time via a
language_id. It is Chatterbox Multilingual v3
fine-tuned jointly on all four languages. Voice cloning is supported: every generation is
conditioned on a reference .wav.
Model summary
| Base | Chatterbox Multilingual v3 (ResembleAI) |
| Fine-tuning | Joint multilingual, t3-only (s3gen and ve frozen) |
| Precision | fp32 |
| Learning rate | 1e-5 |
| Vocabulary | extended to 2458 text tokens (stock is 2454) with per-utterance language tokens [tg] [uz] [kk] [ky] at ids 2454–2457 |
| Steps trained | 20,000 |
| Winner checkpoint | step 10,000 (best joint quality; later checkpoints regress) |
| License | MIT |
| Watermark | Perth (Resemble) watermarked output |
Only the fine-tuned extended t3 (t3_mtl23ls_v2.safetensors) is trained; s3gen.pt,
ve.pt and conds.pt are the frozen Chatterbox components, and
grapheme_mtl_merged_expanded_v1.json is the extended 2458-token tokenizer.
Training data
344,031 utterances total:
| Language | Utts | Source |
|---|---|---|
| Tajik (tg) | 100,000 | audiobook + diarized |
| Uzbek (uz) | 100,000 | uzbekvoice |
| Kazakh (kk) | 100,000 | KSC2 |
| Kyrgyz (ky) | 7,339 clips, oversampled 6× | CommonVoice 17 (ky) + FLEURS (ky) |
FLEURS test sets were held out for tg / uz / kk.
Results (round-trip TTS eval, n = 60 per language)
Generate from held-out text → transcribe with an ASR scorer → compare to reference.
| Language | Metric | This combined model | Specialist (single-lang FT) |
|---|---|---|---|
| Tajik | WER | 11.8% median | 10.7% median |
| Uzbek | WER | 18.2% median | 14.8% median |
| Kazakh | CER | 6.9% median | 7.4% median (combined is better) |
| Kyrgyz | — | not scored (no trusted Kyrgyz ASR scorer yet) | — |
Notes:
- Kazakh is reported as CER (character error rate): Kazakh is agglutinative and a base-whisper WER scorer inflates the number, so CER is the trustworthy metric.
- Uzbek is the weak language in joint training — its Latin-minority script destabilizes relative to the Cyrillic-script Tajik/Kazakh.
Raw evidence is in eval/: per_utt_{tg,uz,kk}_n60.jsonl (raw ref/hyp/wer
per utterance at step 10000) and scores.txt.
Inference
CRITICAL: set
CB_TEXT_VOCAB=2458before loading. This model has 2458 text tokens; without this env the loader assumes 2454 and fails.
import os
os.environ["CB_TEXT_VOCAB"] = "2458" # REQUIRED — extended vocab
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
model = ChatterboxMultilingualTTS.from_local("path/to/model_dir", device="cuda")
# eager-attention workaround
model.t3.tfmr.config._attn_implementation = "eager"
wav = model.generate(
"Матни шумо дар ин ҷо", # your text
language_id="tg", # one of: tg | uz | kk | ky
audio_prompt_path="ref.wav", # reference voice (cloning) — required
)
language_idselects the spoken language:"tg","uz","kk", or"ky".- A reference
.wav(audio_prompt_path) is required — this is a voice-cloning TTS. - Output audio is Perth-watermarked.
Files
t3_mtl23ls_v2.safetensors— fine-tuned extended t3 (trained)s3gen.pt— frozen S3 generatorve.pt— frozen voice encoderconds.pt— default conditioninggrapheme_mtl_merged_expanded_v1.json— extended 2458-token tokenizer ([tg][uz][kk][ky])
License & attributions
- Model: MIT (following Chatterbox).
- Training data attributions: uzbekvoice (CC0), KSC2 (Kazakh Speech Corpus 2), CommonVoice (CC0), FLEURS (CC-BY).