Pocket TTS — Hindi (हिन्दी) · single voice syspin_merged_female_tts_unk

86 MB. A complete Hindi text-to-speech model, 4-bit quantised, running

real-time on a CPU with no GPU anywhere in the loop.

Hear it first — sample_int4_g32.wav is generated by the 4-bit weights in this repo, not by the float model.

~110M parameters, 24 kHz, one voice, MIT.

Built by Prasad Vittaldev. Follow me on LinkedIn — I publish every new voice and language there as it launches.

Why this exists

Kyutai's pocket-tts is a genuinely remarkable piece of work: a text-to-speech model small enough and fast enough to run in real time on a CPU, released under MIT, with a training recipe that anyone can reproduce. That combination is rare — most speech models of this quality assume a GPU at inference and a restrictive licence.

What it shipped with, though, were European languages. India has none.

This repository is that gap closed for Hindi. Kyutai's architecture, training recipe and codec are used as published; what is added here is the part that did not exist — Hindi data preparation, forced alignment in native Hindi script, a Hindi tokenizer, a full teacher finetune, and a depth distillation down to a model this size. The result keeps everything that makes pocket-tts a great deal — CPU real-time, MIT, on-device, no API key, no per-character billing, nothing leaving the machine — and makes it speak Hindi.

Model

The voice is fixed

This model was distilled on a single speaker, and voice cloning did not survive that. It ships a voice_prompt.wav because the API requires one, but supplying a different prompt does not hand you that speaker's voice. Measured across 4 prompts spanning 109-200 Hz, this model's output moved 32 Hz in total, staying between 138 and 170 Hz:

The teacher follows its prompt across the range; this model tracks it only weakly. It cannot be pushed below the speaker it was trained on either — the 109 Hz prompt still comes out at 150 Hz.

That is the expected consequence of single-speaker distillation and it is why each voice is a separate repository rather than one model with selectable voices. If you want a different voice, use a different repo.

Files: start with the 4-bit one

The 4-bit file stores every backbone weight as two 4-bit codes per byte plus one fp16 scale per group of 32, and keeps the Mimi audio codec in fp16 — quantising a neural codec causes artefacts that have nothing to do with the language model. Reconstruction is (code - 8) * scale.

What 4-bit buys and what it does not. It is a download and storage win: 86 MB instead of 418 MB, a 4.8x reduction. It is not a memory win — unpacking restores fp16 tensors, so RAM at inference matches the bf16 build. A native int4 kernel would change that, and none exists for pocket-tts today.

For speed, add --quantize to any of the three: dynamic int8 on attention and feed-forward at load time, measured at 5.3x realtime against 3.4x realtime for float.

How far down can quantization go?

Every level below was generated from this model and listened to. Weight error is the mean relative error introduced in the 85.3 M backbone weights; duration is on one fixed 885-character Telugu passage where the reference is 74.3 s — a short clip means the model dropped text, which is an objective failure that needs no listening.

A sample_*.wav for each row is in this repo, all the same passage, so the degradation is audible rather than merely tabulated.

int4 is the lowest usable precision and the audio holds up. It is not the shipped default for a practical reason: pocket-tts implements int8 only, and torchao's int4 CPU kernels need mslk, so there is no int4 runtime to point you at. quantize_int4.py in this repo reproduces the int4 weights exactly if you have a runtime that can use them. int3 and below are documented so nobody has to repeat the experiment — they lose words.

Note the backbone is not the whole model: the Mimi codec adds ~20 M parameters and is deliberately left in float, because quantizing a neural audio codec introduces artefacts unrelated to the language model.

Usage

pip install pocket-tts

The 86 MB route

Download the 4-bit file, unpack it once, then use it like any other checkpoint:

huggingface-cli download prasadvittaldev/pocket-tts-hindi-female-syspin model_int4_g32.safetensors quantize_int4.py --local-dir .
python quantize_int4.py --unpack --src model_int4_g32.safetensors --dst model.safetensors

That writes a normal safetensors file the stock loader reads. Point a config at it, or just use the hosted config and the full-precision weights below.

Straight from the Hub

python -m pocket_tts generate   --config hf://prasadvittaldev/pocket-tts-hindi-female-syspin/config.yaml   --voice  hf://prasadvittaldev/pocket-tts-hindi-female-syspin/voice_prompt.wav   --text   "गोदावरी नदी के किनारे बसे उस छोटे से गाँव में सूर्यास्त बहुत सुंदर लगता है।"   --output-path out.wav   --device cpu --quantize

--quantize is the int8 path: fastest at runtime, 5.3x realtime.

from pocket_tts import TTSModel
import numpy as np, soundfile as sf

model = TTSModel.load_model(config="hf://prasadvittaldev/pocket-tts-hindi-female-syspin/config.yaml", quantize=True)
model.to("cpu")
state = model.get_state_for_audio_prompt("voice_prompt.wav")
chunks = list(model.generate_audio_stream(
    model_state=state, text_to_generate="गोदावरी नदी के किनारे बसे उस छोटे से गाँव में सूर्यास्त बहुत सुंदर लगता है।"))
sf.write("out.wav", np.concatenate([np.asarray(c).reshape(-1) for c in chunks]), 24000)

Run it in a browser or on a phone (ONNX)

An ONNX bundle is in onnx/. It was exported with KevinAHM/pocket-tts-onnx-export and verified against the PyTorch model tensor-for-tensor, then run end to end through ONNX Runtime and listened to before publishing.

Measured on CPU with 4 threads, same sentence for each:

* 86 MB is the practical browser bundle: this model has a fixed voice, so the 37.9 MB mimi_encoder — which exists only to encode a new voice prompt — can be dropped once the prompt state is precomputed. Keep it if you want to experiment with prompts, though note the voice will not change (see above).

int4 is 4.9x smaller than float32 and ~30% faster. It is not faster than int8, because MatMulNBits dequantises to compute: 4-bit buys download size, and in a browser that is usually the constraint that matters.

Two graphs stay in float32 on purpose. flow_lm_flow is 26 Gemm nodes and zero MatMul, and MatMulNBits only rewrites MatMul; text_conditioner is an embedding table, which weight-only quantisation skips. Neither is worth graph surgery for the few MB involved.

Runtimes that can load this: ONNX Runtime Web for browsers, and sherpa-onnx for Android/iOS/Raspberry Pi with Kotlin, Swift, Java, Dart and C# bindings.

One important note on long text

Generate one sentence at a time and join the results. pocket-tts splits long input internally, but it locates sentence ends using tokens from its own tokenizer, and an Indic tokenizer does not produce the pieces it looks for — for Hindi the boundary search finds nothing and the whole passage is pushed through as a single chunk, which silently drops words. Splitting on . ? ! and the danda । before you call the model avoids this completely.

Sentences that are themselves long need one more split. A generation chunk is capped at 50 tokens and anything past the cap is dropped, so a single sentence of roughly 130 characters or more can lose a clause. It is intermittent — the same sentence can come back whole on the next call — so a short result is the signal to look for. Splitting on commas as well as full stops for anything over about 110 characters avoids it. Measured on six held-out sentences of 133-139 characters: four complete, two missing a clause.

Audio samples

• sample_bf16.wav
• sample_int4_g32.wav
• sample_int8.wav

How it was made

1. Teacher. Kyutai's released 24-layer English model was finetuned on 131 hours of Hindi speech (20,000 steps). The text embedding is reset and relearned against a Hindi sentencepiece tokenizer, so nothing English-language survives in the text side; the English weights serve only as an acoustic starting point.
2. Alignment. Word-level forced alignment in native Hindi script. The obvious choice, Meta's MMS multilingual aligner, is unusable here — it romanises, and its vocabulary contains no Hindi characters at all.
3. Distillation. The 24-layer teacher was distilled down to this 6-layer student (40,000 steps), trained only on this single speaker's 53.4 hours / 21,524 clips. Depth is where the size is: the backbone is ~96% of the parameters.

Credits

This model would not exist without:

• kyutai-labs/pocket-tts — the architecture, the Mimi codec, the training and distillation recipes, released under MIT. All the hard modelling work is theirs.
• Speech corpora:
• SPRINGLab/IndicTTS-Hindi — IndicTTS (SPRING Lab, IIT Madras)
• Ritwika03/syspin_merged_male_tts — SYSPIN (IISc Bangalore)
• Ritwika03/syspin_merged_female_tts — SYSPIN (IISc Bangalore)

Licence

MIT, matching upstream pocket-tts. The training corpora carry their own licences — check the dataset pages above before commercial use.

More voices and languages

More Hindi voices and more Indian languages are on the way. Follow Prasad Vittaldev on LinkedIn for each new model launch.