Chatterbox Nano — CoreML
CoreML export of ResembleAI/chatterbox-nano (110M GPT2-small T3 + distilled 2-step meanflow S3Gen, English, paralinguistic tags) for Apple platforms, converted by FluidInference.
Each model ships as both .mlpackage (source) and compiled .mlmodelc.
Models
| File | Size (fp16) | Role | Compute |
|---|---|---|---|
T3Nano-Prefill-T512-M1536-fp16 | 173 MB | GPT2-small prefill over ≤512-token context (batch 1, no CFG), initializes 1536-slot KV cache | CPU+GPU |
T3Nano-Decode-M1536-fp16 | 184 MB | Single-step AR decode, KV cache via I/O tensors (9 ms/step) | CPU+GPU |
T3Nano-Decode-M1536-fp16-stateful | 184 MB | Single-step AR decode, KV cache in MLState (3.3 ms/step; macOS 15+/iOS 18+) | CPU+GPU |
FlowMean-N500-fp16 | 228 MB | S3Gen meanflow: 500-token bucket → 1000 mel frames, 2 plain Euler steps in-graph (0.38 s/call) | CPU+GPU |
HiFT-T1000-fp16 | 40 MB | HiFTNet vocoder: mel → 24 kHz waveform (0.09 s/call) | CPU+GPU |
FlowMean-N1000-fp16 | 242 MB | S3Gen meanflow, 1000-token bucket → 2000 mel frames (extended ~30 s output; FluidAudio #924) | CPU+GPU |
HiFT-T2000-fp16 | 40 MB | HiFTNet vocoder for the 2000-frame extended bucket | CPU+GPU |
tables/tables.safetensors | 87 MB | text/speech embedding tables (host applies; GPT2 wpe is in-graph) | |
tables/voice-default.safetensors | 0.7 MB | precomputed built-in voice conditioning (T3 cond embeds + S3Gen ref dict) | |
tokenizer/ | 1.4 MB | GPT2 BPE tokenizer (50276 ids incl. 20 paralinguistic tags: [laugh], [chuckle], [sigh], …) |
Output budget / bucket choice
The flow bucket holds voice prompt tokens + generated speech tokens + 3 silence tokens. The built-in voice uses 250 prompt tokens, so N500
yields ≤247 generated tokens ≈ 9.9 s of audio per call and N1000
≤747 ≈ 29.9 s. Likewise the T3 prefill window holds
voice conditioning (376) + text BPE + 1 BOS, i.e. ≤135 text tokens.
Pick ONE flow/vocoder pair at load time; the buckets are static shapes, so
the extended pair roughly doubles flow+vocoder latency even for short
outputs.
The AR stage runs at ~300 tok/s against the 25 Hz speech-token rate
(~12× real-time) with the stateful decode; the full compute path is
~3.4× real-time with I/O-KV decode and ~6× with MLState, measured on an
M-series Mac with .cpuAndGPU.
⚠️ The sibling Multilingual T3 packages hard-crash under .cpuOnly; the
Nano packages have not been matrix-tested there — prefer .cpuAndGPU or
.all.
Samples
samples/ has CoreML end-to-end renders
(e2e_nano_*.wav) next to stock PyTorch renders (baseline_*.wav), all
using the built-in voice. *_tags.wav exercises the [chuckle]
paralinguistic tag. Parakeet-v3 transcribes both e2e renders verbatim.
To synthesize locally without the upstream checkpoint (Apple silicon):
git clone -b feat/chatterbox-nano-coreml https://github.com/FluidInference/mobius
cd mobius/models/tts/chatterbox/coreml
uv sync
uv run python verify/e2e_nano_coreml.py # models auto-download from this repo
Runtime boundary
The graphs cover T3 prefill/decode, the meanflow flow, and the HiFT vocoder. The host runtime must provide:
- text normalization (
punc_norm) + BPE tokenization (tokenizer/; no BOT/EOT wrapping — the prefill context is cond ++ text ++ one BOS speech embed) - embedding prep from
tables.safetensors(no positional add — GPT2wpeis applied in-graph) - turbo sampling: temperature 0.8 → top-k 1000 → top-p 0.95 → repetition penalty 1.2, EOS = 6562 (no CFG, no alignment analyzer)
- SineGen randomness (
phase_vec,noiseinputs to HiFT) and flow noisez - flow bucket padding/cropping; MLState seeding from prefill KV for the stateful decode
Voice cloning from a reference wav additionally needs the VoiceEncoder /
S3TokenizerV2 / CAMPPlus encoders, which are not converted here; voices can
be prepared offline in Python (export-tables-nano.py --ref-wav) and
shipped as voice-*.safetensors.
Parity (vs upstream PyTorch)
| Check | Result |
|---|---|
| T3 wrappers vs stock (fp32) | logits 1.4e-05 |
| T3 CoreML fp16 (I/O-KV / stateful) | logits 2.2e-02 / 1.9e-02 |
| Flow CoreML fp16 | mel max 5.9e-02, mean 1.8e-03 |
| HiFT CoreML fp16 | wav max 3.3e-03, mean 1.2e-04 |
| e2e ASR round-trip | verbatim transcripts; [chuckle] audibly realized |
License
MIT, following upstream ResembleAI/chatterbox-nano. Upstream embeds Resemble's Perth watermarker in its Python pipeline; this CoreML export does not include a watermarking stage.