Qwen3-TTS-12Hz-1.7B-Base — GGUF + ONNX
The official Qwen/Qwen3-TTS-12Hz-1.7B-Base weights, converted to run on llama.cpp and ONNX Runtime — that is, without PyTorch and without Python. Nothing was retrained: these are the same weights in another format, with the talker quantised.
They were made for voicy, a local speech server, whose Rust build runs entirely on them: on an RTX 3080 synthesis goes at ×3.4–3.8 of real time against ×1.4 through PyTorch, and takes 2.4 GB of video memory against 5.6, at the same intelligibility and the same voice similarity (measurements).
Files
| file | what it is | size |
|---|---|---|
qwen3_tts_talker.q5_k.gguf | talker, quantised to q5_k — what voicy uses by default | 1.0 GB |
qwen3_tts_talker.q8_0.gguf | the same, q8_0 | 1.5 GB |
qwen3_tts_talker.f16.gguf | the same, unquantised | 2.8 GB |
qwen3_tts_predictor.q8_0.gguf | codec predictor, q8_0 | 151 MB |
qwen3_tts_decoder.fp16.onnx | codec decoder, converted to fp16 | 230 MB |
qwen3_tts_codec_encoder.fp32.onnx (+ .data) | codec encoder for the reference clip | 226 MB |
qwen3_tts_speaker_encoder.fp32.onnx (+ .data) | speaker encoder | 48 MB |
embeddings/ | codec embedding tables, .npy | 866 MB |
tokenizer.json | the model's tokenizer | 11 MB |
Only the q5_k talker is needed for the default setup; the other two are there for comparison.
How they behave
RTX 3080 10 GB, ten sentences × three seeds, the turgenev voice. CER — by
transcribing the result back with Whisper large-v3-turbo; similarity — cosine
between the speaker-encoder vectors of the reference clip and of the synthesis.
| CER | speed | video memory | similarity | |
|---|---|---|---|---|
| PyTorch, original weights | 2.1% | ×1.42 | 5.6 GB | 0.985 |
| this, f16 talker | 2.1% | ×2.65 | 4.5 GB | 0.986 |
| this, q8_0 talker | 2.2% | ×3.12 | 2.6 GB | 0.986 |
| this, q5_k talker | 2.1% | ×2.96 | 2.2 GB | 0.986 |
The speeds above are of the Python pipeline used for the comparison; voicy's Rust engine runs the same q5_k files at ×3.39 in the same test. Bit-identical audio to PyTorch is not possible and is not claimed — different kernels, other orders of summation — but intelligibility, voice similarity, pauses and pace come out the same.
Use
With voicy, this is the whole of it:
voicy setup # libraries and the other models
voicy serve --port 8080
Point it elsewhere with VOICY_TTS_GGUF_DIR, and pick another talker with
TTS_GGUF_TALKER=q8_0|f16.
Outside voicy, the files are ordinary llama.cpp and ONNX Runtime inputs, but
the pipeline around them — the prompt of the talker, the codec predictor, the
codebook tables — is not a standard one; the shortest description of it is the
implementation: rust/src/native/qwen.rs.
How they were made
The conversion scripts are HaujetZhao/Qwen3-TTS-GGUF
(MIT) at commit 74feb581bc8cefb835fc608107e857036d7580a1: the talker and the
predictor to GGUF, the codec and the speaker encoder to ONNX, the codec
embeddings to .npy. Then, in voicy's scripts/convert_qwen.py:
llama-quantize(llama.cpp b11090) — the talker to Q5_K and the predictor to Q8_0;- the codec decoder to fp16 with
onnxruntime.transformers.float16, keepingLayerNormalization,SoftmaxandRangein fp32.
Changes against the original model: format and precision only. No weights were trained, tuned or edited.
Licence
Apache-2.0, from Qwen3-TTS-12Hz-1.7B-Base;
LICENSE is included. The conversion scripts are MIT. Whatever the licence
allows for the original weights, it allows for these; the responsibility for
what is synthesised — voice cloning included — stays with whoever synthesises.
По-русски
Официальные веса Qwen3-TTS-12Hz-1.7B-Base, переведённые в GGUF и ONNX, чтобы синтез шёл на llama.cpp и ONNX Runtime — без PyTorch и без Python. Модель не переобучалась: те же веса в другом формате, говорящая часть квантована.
Сделано для voicy: на RTX 3080 синтез идёт
×3.4–3.8 к реальному времени против ×1.4 через PyTorch и занимает 2.4 ГБ
видеопамяти против 5.6 — при той же разборчивости (CER 2.1%) и том же сходстве
голоса (0.986). По умолчанию нужен только qwen3_tts_talker.q5_k.gguf;
q8_0 и f16 лежат рядом для сравнения.
Установка — voicy setup, дальше voicy serve. Свой каталог задаётся через
VOICY_TTS_GGUF_DIR, вариант говорящей части — TTS_GGUF_TALKER.