OmniVoice, int8 + fp16, for MLX
k2-fsa/OmniVoice converted for omnivoice-mlx, a torch-free MLX port for Apple silicon. The Qwen3 backbone's Linear layers are quantised to int8 with group size 64, and activations are fp16. The eight codebook output heads are left unquantised in fp32: quantising them buys no speed and costs naturalness (UTMOS 2.835 → 2.794). 829 MB, 1.6 GB resident with the codec.
| source | framework | device | weights | activations | steps | RTF | notes |
|---|---|---|---|---|---|---|---|
| official | torch | CPU | fp32 | fp32 | 32 | ≈2–3 | |
| official | torch | MPS | fp32 | fp32 | 32 | 0.98 | |
| mlx-audio | MLX | GPU | bf16 | bf16 | 32 | 0.68 | |
| this | MLX | GPU | int8 | fp16 | 32 | 0.381 | |
| this | MLX | GPU | int8 | fp16 | 16 | 0.106 | default |
| this | MLX | GPU | int8 | fp16 | 16 | 0.080 | batched B=4–8 |
CER, speaker similarity and UTMOS are unchanged from the official 32-step sampler. M2 Max 12-core CPU / 38-core GPU / 32 GB, macOS 26.6, MLX 0.32.2. Method, ablations and dead ends are in the port's research log (Chinese).
Use
pip install git+https://github.com/Remember2015/omnivoice-mlx
hf download remember2015/omnivoice-mlx-q8-fp16 --local-dir models/mlx-q8-fp16
The codec weights have their own licence and are not in this repository; take them from upstream:
hf download k2-fsa/OmniVoice --local-dir models/k2-fsa-OmniVoice
ln -s ../k2-fsa-OmniVoice/audio_tokenizer models/mlx-q8-fp16/audio_tokenizer
python -c "from omnivoice_mlx.codec import write_slim_decoder; write_slim_decoder('models/mlx-q8-fp16', 'float16')"
Decoding reads only that 44 MB branch; the full tokenizer is loaded when encoding a reference clip.
from omnivoice_mlx import OmniVoiceTTS
tts = OmniVoiceTTS("models/mlx-q8-fp16")
voice = tts.make_prompt("my-voice.wav", "exactly what that clip says.")
r = tts.generate("今天天气不错，我们出去走走吧。", voice) # r.audio: float32 24 kHz, r.rtf
What is in here
| file | what |
|---|---|
model.safetensors | int8 g64 backbone, fp16 activations, output heads in fp32 |
config.json | upstream config plus dtype, head_dtype, quantization |
tokenizer.json, tokenizer_config.json | the Qwen3 text tokenizer, unchanged from upstream |
bf16, fp16 and fp32 builds are one minute of scripts/convert.py in the port's repository; only the recommended
configuration is published here.
Licence
| what | licence |
|---|---|
| these files | CC-BY-NC-4.0, inherited from k2-fsa/OmniVoice; non-commercial, attribution required |
| the conversion code | Apache-2.0, see omnivoice-mlx |
| Higgs codec weights | Boson Higgs Audio 2 Community License; not redistributed here, download from upstream as shown above |