DACVAE-TTS Turkish (width 512, 66.5M)
Zero-shot voice-cloning text-to-speech for Turkish, trained from scratch with
dacvae-tts in the latent space of the frozen Meta DACVAE codec
(facebook/dacvae-watermarked, 48 kHz, 128 channels, 25 latent frames/s). Text in (UTF-8 bytes, Turkish number
normalization), 3–15 s reference audio + its transcript in, 48 kHz speech in the reference voice out.
- Architecture: 12 DiT blocks, width 512, 8 heads, single-frame latent tokens, RoPE self-attention, length-aware RoPE text cross-attention, QK-norm, shared low-rank AdaLN (rank 64), byte text encoder (4 conv + 4 self-attention blocks), EDM-style flow-matching target, auxiliary CTC head, skip/repeat contrastive negatives, Muon optimizer, EMA 0.9999.
- Training data:
Vyvo/tr-dataset-12(Turkish podcast segments, 93 h) filtered to quality ≥ 55, Whisper-large-v3 CER ≤ 0.10 and DNSMOS OVRL ≥ 2.8 → 32k clips (~70 h). 60k updates, batch expansion 4, one RTX 4090 (9.5 h). - Checkpoint selection: by Freya-TR-Eval only (495 conversational sentences that are not in the training data,
synthesized with 24 held-out speaker prompts), to exclude checkpoints that memorize the training set. This checkpoint
(
tr-w512-clean, update 60k) had the best Freya score of all runs; an in-domain fine-tune that looked better on the training-domain monitor regressed on Freya and was not published.
Results (Freya-TR-Eval, faster-whisper large-v3 beam 5, Turkish normalization, guidance 5, 32 Euler steps)
| Model | Params | WER % | CER % |
|---|---|---|---|
| Piper (tr) — from the FreyaTTS paper | – | 4.4 | – |
| this model | 66.5M | 4.3 | 2.5 |
| MMS-TTS — paper | – | 6.8 | – |
| FreyaTTS — paper | 183M | 8.0 | 3.0 |
| XTTS-v2 — paper | 467M | 11.1 | – |
| F5-TTS (tr fine-tune) — paper | 336M | 24.3 | – |
Also: speaker similarity 0.946 (WavLM-base-plus-SV cosine vs the prompt), DNSMOS OVRL 2.89 (real podcast speech through the
codec: 3.27). Held-out-speaker cross-utterance cases (48): WER 0.099 / CER 0.053 at guidance 5. Generated audio for every
evaluated checkpoint, with texts and transcripts, is in VoiceHub/dacvae-tts-tr-w512-clean (dataset).
Recommended inference settings (validated on Freya-TR-Eval)
The numbers above use the plain prompt-rate duration rule and one sample per sentence. Inference-side improvements, each measured on Freya-TR-Eval with two disjoint draws of the 24 unseen prompt voices (prompt seed 42 = the set of the table above, seed 1000 = a second set):
| Setting (same checkpoint, guidance 5, 32 steps) | WER % (seed 42) | CER % (seed 42) | WER % (seed 1000) | CER % (seed 1000) |
|---|---|---|---|---|
| prompt-rate rule (published numbers) | 4.32 | 2.50 | 4.40 | 2.20 |
| fixed 15 characters/s (old demo option) | 5.96 | 3.39 | – | – |
duration_mode="clamp" (slow down prompts faster than 17 chars/s) | 3.61 | 2.03 | 3.73 | 1.99 |
duration_mode="predictor" (fitted duration predictor) | 4.14 | 2.26 | 3.99 | 2.39 |
duration_mode="auto" (predictor below 13, clamp above 17 chars/s) | 3.61 | 1.81 | 3.50 | 1.91 |
auto + best-of-3 (Whisper-turbo picks the candidate; demo default) | 1.59 | 0.72 | 1.92 | 0.72 |
Guidance variants did not beat plain CFG on intelligibility: APG (η 0.5, momentum −0.3) and CFG-rescale (φ 0.7) remove most
of the output saturation (files reaching the decoder's tanh ceiling 96% → 58% / 22%, loudness −13.9 → −16.3 / −18.4 LUFS)
and raise speaker similarity/DNSMOS slightly, but WER rises to 4.96% / 5.16%. Independent text/speaker guidance (speaker 3 or 7 with text 5) and interval CFG (t < 0.7) did not help either
(WER 4.63 / 5.14 / 4.42). Best-of-3 is ranked by Whisper-large-v3-turbo and scored by Whisper-large-v3; turbo is distilled
from large-v3, so part of the gain may reflect shared ASR preferences. Speaker similarity (0.947) and DNSMOS (2.93) are
unchanged by the recommended settings.
All runs, per-sentence transcripts and scores: VoiceHub/dacvae-tts-tr-w512-clean, folder demo-experiments/.
Usage
pip install torch torchaudio numpy soundfile scipy pyloudnorm huggingface_hub transformers \
"dacvae @ git+https://github.com/facebookresearch/dacvae.git@414c20785fc3a28373073ea8ef7a1316eeeaca6e"
import sys
from huggingface_hub import snapshot_download
repo = snapshot_download("VoiceHub/dacvae-tts-tr-w512")
sys.path.insert(0, repo) # the `dacvae_tts` package (Turkish frontend, duration modes) is bundled here
from dacvae_tts.frontend import speakable, split_sentences
from dacvae_tts.inference import Synthesizer
tts = Synthesizer(f"{repo}/model.pt", device="cuda") # downloads facebook/dacvae-watermarked on first use
voice = tts.prepare_reference("prompt.wav", "Prompt kaydında söylenen cümlenin tam metni.") # 3-15 s, one speaker
text, changes = speakable("Toplantı 23.09.2026'da saat 14:30'da; bütçe %20 arttı.") # dates, times, symbols -> words
results, info = tts.synthesize_many(
[chunk for chunk, _ in split_sentences(text)], voice, candidates=3, # sentence chunks x candidates, one batch
duration_mode="auto", guidance=5.0, steps=32, sway=-1.0,
)
# results[i][k]["audio"] is candidate k of chunk i (48 kHz float32); rank the candidates with an ASR (the demo uses
# openai/whisper-large-v3-turbo and keeps the lowest CER), then join the chunks.
Synthesizer.synthesize(text, ref_audio=..., reference_text=..., duration_mode="auto", guidance=5.0, steps=32) is the
one-call API for a single sentence. Without reference_text the prompt is transcribed with Whisper (asr_language="tr").
Sampler options: guidance_until/guidance_from (interval CFG), cfg_rescale, apg_eta/apg_norm/apg_momentum,
speaker_guidance (independent text/speaker guidance).
Limitations
- 48 kHz output, but the training audio is podcast MP3 (mostly 12–16 kHz bandwidth); DNSMOS ≈ 2.9.
- Remaining errors: occasional repeated or swapped words (best-of-N removes most of them), letter errors in rare foreign names; quality depends on the prompt (per-voice Freya WER 0.6–9.6 %; fast and noisy prompts are the hardest).
- High guidance (5) saturates the decoder output (loudness ~−14 LUFS, samples at the tanh ceiling in most files); the demo normalizes to −16 LUFS; APG avoids the saturation at some cost in WER.
- Reference audio needs its transcript (typed or via Whisper); reference and target should be different sentences.
- Non-commercial: the training data (
Vyvo/tr-dataset-12, CC-BY-4.0 podcasts) and this model are released CC-BY-NC-4.0; the DACVAE codec keeps its upstream license.
Demo Space
Live demo (ZeroGPU): https://huggingface.co/spaces/Vyvo/dacvae-tts-tr-demo
- Synthesis with the settings above (automatic speaking rate, best-of-3, long text split into sentences, numbers/dates/ currencies/abbreviations read out), Whisper transcript of the reference filled in and editable, WER/CER, speaker similarity and DNSMOS for every output, −16 LUFS 48 kHz WAV.
- Research tools: any checkpoint from the Hub (log in for private repositories — your own permission is used), A/B
comparison of 2–4 checkpoints, batch test on sentence lists or Freya-TR-Eval with a zip of all outputs, API access
(
gradio_client, endpoint/synthesize).
The demo's code is in space/ (app.py interface, engine.py models and ZeroGPU jobs; the dacvae_tts package at the
repository root is reused); space/SPACE.md describes deploying a copy.
Files
model.pt: EMA weights + config + codec metadata + latent normalization (torch.load(..., weights_only=True)).config.json: training/model configuration.dacvae_tts/: inference code (snapshot of the training repository, branchturkish-tts): Turkish text frontend (frontend.py), duration rules and the fitted predictor (duration.py,duration_tr.json), sampler options, batched synthesis (inference.py), output post-processing (audio.py).space/: the Gradio demo.