Twi / Ghanaian English TTS (IPA-driven)

A Piper VITS voice for Twi that also reads the English words Ghanaian text is full of. It is driven by IPA phonemes rather than orthography, which is what lets one model serve both languages from a shared inventory.

Warm-started from en_US/libritts_r/medium and finetuned on 327 h (164 h Twi + 164 h Ghanaian English), 22.05 kHz, 1,555 speaker embeddings.

Use it

pip install stable-twi-tts && pip install "stable-twi-tts[twi]" && apt install espeak-ng
stable-twi-tts --model . --voice twi-6 --text "Akwaaba, wo ho te sen?" --out hello.wav
stable-twi-tts --model . --language mixed --voice twi-1 \
    --text "Mepe se mesua [computer science] wo [University of Ghana]." --out mixed.wav

Code: https://github.com/GhanaNLP/stable-twi-tts

Voices are ranked by measurement, not by training hours

12 voices are exposed of 1,555 speaker embeddings. Each synthesised the same held-out text, which was re-recognised and scored for phoneme error -- Twi with ghana-speech-phoneme-asr, the English spans with KoelLabs/xlsr-english-01.

The two rankings disagree sharply. twi-1 is the best code-switch voice yet 21st of 30 on pure Twi. Two of the three best Twi voices have under 3.3 h of audio each and would never have survived an hours-based shortlist. voices.json records every measurement.

Quality, honestly

Round-trip phoneme error against a real-audio floor, on held-out text:

Twi is good; English is markedly weaker. English had a third of the utterances (43k vs 151k) at 3.5x the length, and VITS learns alignment per utterance, so English got far less practice on harder material. English audio was also 16 kHz upsampled to 22.05 kHz, so English voices are band-limited to 8 kHz and sound duller. Twi was 24 kHz and is unaffected.

Training

Warm-started from en_US/libritts_r/medium and finetuned for 10 epochs / 52,039 steps, about 21.5 hours on a single H200. Batch 32, bf16-mixed, single device, 178,833 training utterances across 1,555 speakers. The shipped weights are epoch 7.

Per-validation metrics

Validation ran twice per epoch. val_mos is UTMOS, a learned naturalness predictor — higher is better. Everything else is a loss — lower is better.

Training losses

Reading these numbers

val_loss is not a quality signal here, and treating it as one would have been a mistake. It starts at 38.08 and ends at 38.04 — flat across the entire run, with a 42.69 spike at epoch 3 that means nothing. That is expected for a GAN: the generator's total loss is measured against a discriminator that is improving alongside it, so the number tracks the balance between the two, not how good the audio is. A run that looked converged at step 2,000 by val_loss in fact kept improving for another 40,000 steps.

The two signals that did carry information disagreed with each other:

• val_mel (mel-spectrogram L1) bottomed at 0.4686 at epoch 8, having drifted down from 0.4792. A 2.2% total improvement — real, but small.
• val_mos peaked at 3.018 at epoch 7 and never recovered: 2.90, 2.76, 2.75, 2.88 after. It is also noisy — epoch 3 swings 2.695 → 2.853 within one epoch — so no single validation is trustworthy on its own.

Epoch 7 was shipped on val_mos plus listening, not on val_mel. Epoch 8's marginally better mel did not sound better. Where a spectral distance and a naturalness predictor disagree by this little, the honest reading is that the model plateaued somewhere around epoch 6–7 and the rest is noise.

The decisive check was neither: a round-trip evaluation — synthesise held-out text, re-recognise it with a phoneme ASR, score the phoneme error against the error the ASR makes on real audio of the same text. That is the "Quality, honestly" table above, and it is the number to trust, because it measures whether the audio says the right thing rather than whether it resembles a reference spectrogram.

Twi was essentially converged by epoch 1; the remaining 8 epochs bought English (−4.2 points), which is the harder half. The Twi number moving up 2.2 points while listening quality clearly improved is a reminder that a 2-point UER difference is within this instrument's noise.

Training was stopped mid-epoch 9, at step 3,459 of 5,560. Nothing was still improving: val_mos had been below its epoch-7 peak for four consecutive validations, and val_mel's best was a checkpoint that sounded no better. Further epochs would have risked overfitting 1,555 pseudo-speakers without a metric able to tell us it was happening.

Finetuning

finetune/ has the Lightning checkpoint, the phoneme id map, the training command and dataset stats. The phoneme id map is required -- the weights encode "id 26 means /n/", so loading them against a different map yields nonsense. See finetune/FINETUNING.md, which also documents two traps: resize the speaker table before loading, and clear the phoneme cache or stale targets are silently reused.

Limits

• Speakers are derived, not real identities -- pseudo-speakers from clustering unlabelled broadcast audio. One person may appear as two voices, and no voice is a consented identity.
• Trained on broadcast speech, so it suits news-register text better than conversation.
• Training targets came from an ASR, not human annotation, at roughly 17% phoneme error.
• cc-by-nc-4.0: non-commercial, inherited from the Twi source audio.