kova-tts-1

[!IMPORTANT] Kova TTS 1 is available under the Research and Non-Commercial Model License. Commercial use requires a separate written commercial license from Kova AI. See License and attribution.

📚 The story behind Kova TTS 1

Kova started as an AI audiobook app with a simple dream: let anyone hear their favorite book read by a voice they love. As we grew to 10,000 readers, we needed to synthesize more than a million minutes of audio, and none of the models available fit what we needed. They were too expensive at that scale, didn't sound right for long-form narration, or couldn't be used commercially. So we decided to build our own.

We trained Kova TTS 1 specifically for audiobook reading: warm, natural, and pleasant to listen to for hours at a time. Some of our favorite memories from this project are of voice actors coming into the office to record with us, and the moment we first heard their cloned voices narrating a chapter they had never read aloud. Training this model was a blast from start to finish, and today we're excited to share it with the community.

The licensed voice actor voices we use on kova.ai can't be published here, but we've documented exactly how to clone a voice yourself, so you can recreate that same special experience: your book, read by the voice of your dreams ✨

📖 Overview

Kova TTS 1 is an English text-to-speech model built on Llama 3.2 1B, with streaming generation, voice cloning, and 48 kHz mono audio output. Run it locally with the Kova TTS runtime, use one of five pretrained voice adapters, or fine-tune a voice of your own.

📘 Documentation · 🗣️ Pretrained voices

✨ Highlights

• 📚 Built for Narration — Trained for audiobook reading: warm, natural, and pleasant to listen to for hours.
• 🎙️ Voice Cloning — Clone a voice from reference audio and its matching transcript, with no fine-tuning required.
• 🔊 48 kHz Output — High-fidelity mono audio from a neural codec with a 48 kHz decoder.
• 📢 16 and 32 kHz Inputs — Naturally upsamples low-quality audio while maintaining the highest fidelity
• ⏱️ Word Timestamps — Align audio with text for read-along highlighting, subtitles, and precise editing.
• 🔢 Text Normalization — Numbers, abbreviations, and unusual spellings are expanded automatically with the normalize extra, so text can be passed in as written.
• ⚡ Low Latency — ~190 ms warm time to first audio on an RTX 5090 with the local runtime, and under 100 ms in our optimized H100 serving system.
• 🌊 Streaming — Stream from Python, HTTP Server-Sent Events, or WebSocket.
• 🧩 Pretrained Voices — Five optional LoRA voice adapters, or fine-tune your own.
• 💻 Runs Locally — NVIDIA CUDA, Apple Silicon through MLX, or CPU.

📊 Model at a glance

This is a model and local inference release. The included server is intended for local use, experimentation, and integration development; it is not a production server release. See Performance and deployment.

🎧 Try it

📝 "Originally, we built our own voice model for internal use because we couldn't find one that made the unit economics work. Over time, we realized the technology had potential far beyond our own needs. So, we decided to make it public, hoping it could unlock entirely new categories of businesses and experiences that simply haven't been possible at current industry pricing."

Note: these voices are from our production API, which we do not have permission to release. The demo provides some publicly available voices as reference.

Experiment in the hosted demo, or start the local browser demo after installing the runtime below:

uv sync --extra demo
uv run kova-tts demo --preload

The local demo supports streaming playback, installed voices, and voice cloning.

🚀 Quickstart

Requires Python 3.10+ and uv. A CUDA GPU is recommended for interactive generation. Install the runtime from the source repository:

git clone https://github.com/evalabs-ai/kova-tts
cd kova-tts
uv sync
uv run kova-tts download
uv run kova-tts generate "Hello world." --out out.wav

kova-ai/kova-tts-1 is the runtime's default model repository. The base model and codec also download automatically on first use. See the installation guide for hardware requirements, local paths, and authentication for private or gated access.

From Python in the same environment:

from kova_tts import KovaTTS

tts = KovaTTS.from_pretrained()
wav = tts.generate("Hello world.")  # float32 mono audio at 48 kHz
tts.save(wav, "out.wav")

🎙️ Voice cloning and pretrained voices

Clone a voice using a reference recording and a transcript that matches it word for word:

voice = tts.clone("reference.wav", transcript="Exactly what the reference clip says.")
wav = tts.generate("A new sentence in the reference voice.", voice)
tts.save(wav, "cloned.wav")

Reference files are encoded at 32 kHz, or natively at 16 kHz when the recording is below 32 kHz (phone audio, most speech datasets). The decoder produces 48 kHz output. See the voice cloning guide for reference-audio preparation and optional automatic transcription. You are responsible for having the rights to the voice you clone.

For ready-to-use voices, download the five pretrained LoRA adapters: bdl, slt, jmk, awb, and Kathleen. The base model also works without an adapter. To train your own adapter, see the fine-tuning guide.

⚡ Performance and deployment

The accompanying runtime documents these measurements on an RTX 5090, at batch 1 with bfloat16 and a 4,096-token KV cache:

Model decode throughput measures audio-token generation; it is not a measurement of end-to-end serving capacity. First-use model loading and codec warm-up add latency. See the benchmark context.

With production serving optimizations on NVIDIA H100 GPUs, Kova achieves under 100 ms time to first audio in our optimized serving system. The local runtime included in this release is a separate implementation with the measurements shown above.

The local server processes one utterance at a time and has no built-in authentication. A production deployment requires its own access control, concurrency management, and serving optimizations. The hosted service at kova.ai is operated separately from this local runtime.

⚠️ Limitations

• 🌍 English speech generation; other languages are not supported by this release.
• 🎚️ Voice cloning depends on reference-audio quality and an accurate transcript.
• 🖥️ Generation speed and first-audio latency depend on hardware, warm-up, input text, and the serving configuration.

🏗️ Architecture and model files

The Llama backbone predicts discrete audio tokens, and a neural codec converts them to speech. The codec uses an 80-token/second representation with a 48 kHz decoder. See the architecture guide for the token layout, codec, and streaming implementation.

The model configuration, weights, and tokenizer files live at the repository root. codec.pt is the audio codec checkpoint used to turn generated codes into speech. alignment.pt is the 13.7M-parameter CTC aligner behind word timestamps; it reads the generated codes, not the audio. WavLM is fetched separately from microsoft/wavlm-large when encoding audio for voice cloning; base speech generation does not require it.

For transcript alignment, a secondary 13.7M-parameter CTC model aligns the audio tokens directly with the text's characters.

📜 License and attribution

Use is governed by the Research and Non-Commercial Model License. Commercial use requires a separate written commercial license from Kova AI. Third-party components retain their own license terms; see NOTICE and the full upstream license texts.

The pretrained LoRA voices also have a Voice Package Supplement and a dataset NOTICE, both included in the voice repository.

Built with Kova TTS. Built with Llama.

📬 License questions: legal@evalabs.ai.