Chatterbox Multilingual — CoreML
CoreML export of ResembleAI/chatterbox
multilingual (23 languages, t3_mtl23ls_v2 + s3gen) for Apple platforms,
converted by FluidInference
(conversion toolkit: mobius PR #89).
Each model ships as both .mlpackage (source) and compiled .mlmodelc.
Models
| File | Size (fp16) | Role | Compute |
|---|---|---|---|
T3-Prefill-T256-M1024-fp16 | 977 MB | Llama-520M prefill over ≤256-token context (CFG batch 2), initializes 1024-slot KV cache | CPU+GPU |
T3-Decode-M1024-fp16 | 977 MB | Single-step AR decode, KV cache via I/O tensors (38 ms/step) | CPU+GPU |
T3-Decode-M1024-fp16-stateful | 977 MB | Single-step AR decode, KV cache in MLState (16.8 ms/step; macOS 15+/iOS 18+) | CPU+GPU |
Flow-N500-fp16 | 229 MB | S3Gen flow: 500-token bucket → 1000 mel frames, 10-step CFG Euler in-graph | CPU+GPU |
HiFT-T1000-fp16 | 40 MB | HiFTNet vocoder: mel → 24 kHz waveform (0.09 s/call) | CPU+GPU |
tables/tables.safetensors | 34 MB | text/speech embedding + learned positional tables (host applies) | |
tables/voice-default.safetensors | 0.1 MB | precomputed built-in voice conditioning (T3 cond embeds + S3Gen ref dict) | |
tokenizer/grapheme_mtl_merged_expanded_v1.json | 23-language grapheme tokenizer |
⚠️ Do not load the T3 packages with .cpuOnly — prediction hard-crashes
(also independently reported by other Chatterbox CoreML ports). Use
.cpuAndGPU or .all.
Samples
samples/ has CoreML end-to-end renders (e2e_*.wav)
next to stock PyTorch renders (baseline_*.wav) for en/de/fr, all using the
built-in voice.
To synthesize locally without the upstream checkpoint (Apple silicon):
git clone -b feat/chatterbox-mtl-coreml https://github.com/FluidInference/mobius
cd mobius/models/tts/chatterbox/coreml
uv sync
uv run python verify/e2e_coreml.py --lang en # models auto-download from this repo
Runtime boundary
The graphs cover T3 prefill/decode (with the multilingual alignment-analyzer attention rows as outputs), the S3Gen flow, and the HiFT vocoder. The host runtime must provide:
- text normalization + tokenization (
tokenizer/) - embedding prep from
tables.safetensors(text/speech + positional; the stock prefill context ends with two BOS embeds — replicate exactly) - CFG combine
cond + w*(cond-uncond), repetition penalty, min-p/top-p, sampling, EOS handling - the
AlignmentStreamAnalyzerheuristics, fed by the exportedalign_attnrows (reference port:verify/analyzer_port.pyin the conversion toolkit) - SineGen randomness (
phase_vec,noiseinputs to HiFT) and CFM noisez - flow bucket padding/cropping; MLState seeding from prefill KV for the stateful decode
Voice cloning from a reference wav additionally needs the VoiceEncoder /
S3TokenizerV2 / CAMPPlus encoders, which are not converted here; voices can
be prepared offline in Python (export-tables.py --ref-wav) and shipped as
voice-*.safetensors.
Parity (vs upstream PyTorch)
| Check | Result |
|---|---|
| T3 wrappers vs stock (fp32) | logits 3.8e-05, alignment rows exact |
| T3 CoreML fp16 | logits 2.6e-02 (range ±15), align 1.4e-03 |
| Flow CoreML fp16 | mel max 2.4e-02, mean 2.7e-03 |
| HiFT CoreML fp16 | wav max 1.7e-02, mean 2.6e-04 |
| e2e ASR round-trip (en) | exact transcript, matches PyTorch baseline |
License
MIT, following upstream ResembleAI/chatterbox. Upstream embeds Resemble's Perth watermarker in its Python pipeline; this CoreML export does not include a watermarking stage.