PocketTTS CoreML
CoreML conversion of kyutai/pocket-tts for on-device inference on Apple platforms.
Models
| Model | Description | Size |
|---|---|---|
| cond_step | KV cache prefill (voice + text conditioning) | ~200MB |
| flowlm_step | Autoregressive generation (transformer_out + EOS) | ~200MB |
| flow_decoder | Flow matching denoiser (8 Euler steps per frame) | ~190MB |
| mimi_decoder | Streaming audio codec (1920 samples per frame) | ~11MB |
Voices
4 pre-encoded voices in constants_bin/:
alba(default),azelma,cosette,javert
Voice cloning is supported: the Mimi encoders are included (mimi_encoder*.mlmodelc at the root for English, v2.1/<lang>/mimi_encoderv3.mlmodelc per language pack). In FluidAudio, use PocketTtsManager.cloneVoice() or fluidaudiocli tts "…" --backend pocket --clone-voice speaker.wav.
Usage
import FluidAudioTTS
let manager = PocketTtsManager()
try await manager.initialize()
let audio = try await manager.synthesize(text: "Hello, world!")
See https://github.com/FluidInference/FluidAudio for the full Swift framework.
License
CC-BY-4.0, inherited from https://huggingface.co/kyutai/pocket-tts. Attribution to Kyutai is required.
References
- https://huggingface.co/kyutai/pocket-tts
- https://arxiv.org/abs/2410.00037
- https://github.com/FluidInference/FluidAudio