Kokoro-Thai INT8 (Refined Q8 ONNX)
This repository provides an optimized, high-fidelity INT8 quantized ONNX checkpoint for the Thai Kokoro TTS model (FastThaiG2P lineage).
Optimized and packaged for Text To Speech on Steam.
Key Highlights
- 33% Smaller Disk Footprint: Reduced from 310.5 MB (FP32) down to 208.4 MB (INT8).
- 22.5% Lower RAM Usage: Saves 105.3 MB of system RAM during load and reduces peak working-set memory by 118.1 MB.
- 100% Timing Parity (0.0 ms Duration Drift): Evaluated across 534 rigorous test cases spanning all 44 Thai consonants, digits (0–9, ๐–๙), minimal tone pairs, decimals, currency formats, multi-lingual English loanwords, and massive 475-token paragraphs. The duration predictor matched FP32 timing with exact 0-frame drift (0.0 ms) across all 534 cases.
- Tonal Pitch Integrity: Mean fundamental frequency ($F_0$) Pearson correlation $r = 0.9907$ with pitch RMSE $< 4.2\text{ Hz}$ (well below human ear pitch discrimination thresholds of $3\text{--}5\text{ Hz}$).
- Whisper ASR Verified: 100% transcription parity with baseline FP32 when verified on local Whisper large-v3-turbo.
- Native CPU Acceleration: Optimized for x86_64 AVX2 / VNNI execution on the standard ONNX Runtime CPU Execution Provider.
Quantization Architecture & Policy
Naive static quantization of Kokoro typically ruins duration modeling and introduces vocoder distortion. This model follows the upstream Kokoro quantization policy:
| Component | Precision | Rationale |
|---|---|---|
Thai BERT Encoder (/bert) | FP32 | Preserves semantic attention and prevents sequence length $>200$ overflow/NaN crashes. |
| Duration & Pitch LSTMs | FP32 | Eliminates duration drift and preserves Thai 5-tone contour trajectories. |
Vocoder Upsamplers (/decoder/generator/ups) | FP32 | Prevents metallic noise and artifacts in HiFi-GAN transposed convolutions. |
| Text Encoder CNNs & Projections | INT8 | Quantized weights and activations via symmetric static calibration. |
| AdaIN Linear Layers | INT8 | Accelerated style conditioning with low computational overhead. |
Benchmark Profile (ONNX Runtime CPU - 4 Threads)
| Metric | Baseline FP32 | Refined Q8 | Delta / Improvement |
|---|---|---|---|
| Model File Size | 310.5 MB | 208.4 MB | -102.1 MB (-32.9%) |
| RAM Footprint (Delta) | 466.9 MB | 361.6 MB | -105.3 MB (-22.5%) |
| Peak Working Set RAM | 742.8 MB | 624.7 MB | -118.1 MB |
| Cold TTFA (Load + First Sound) | 1,199.0 ms | 1,354.8 ms | ~1.35s from cold launch |
| Warm Latency (Very Short: 12 tok) | 246.8 ms | 232.4 ms | +5.9% faster (4.4x real-time) |
| Warm Latency (Short: 32 tok) | 445.3 ms | 433.7 ms | +2.6% faster (4.3x real-time) |
| Warm Latency (Medium: 98 tok) | 935.0 ms | 929.9 ms | +0.5% faster (4.2x real-time) |
| Warm Latency (Long: 268 tok) | 2,165.4 ms | 2,138.2 ms | +1.3% faster (3.9x real-time) |
| Warm Latency (Very Long: 475 tok) | 3,417.5 ms | 3,384.8 ms | +1.0% faster (3.8x real-time) |
| Duration Drift vs FP32 | 0.0 ms | 0.0 ms (Exact 0 frames) | 100% Timing Match |
| Log-Spectral Distance (LSD) | 0.0 dB | 5.0 dB | Pristine audio fidelity |
File Manifest & Integrity
| File | Size | SHA-256 Checksum |
|---|---|---|
kokoro_thai_q8.onnx | 208.4 MB | D5372F00192E42A334A53FE91EC71E42F11C7FDB6D6667578073698A09235CA1 |
config.json | 2.5 KB | EEAC00CF7E2452296DC26998AE71CEBDBF5BF532F3402C93A7F13ACD9847F10E |
thai_som.npy | 522.4 KB | 0E4B0AAAE4BECE614745E5E94849301046A0BFA3F39B6F3EBC152A7DCE598C7F |
Usage Example (Python ONNX Runtime)
import onnxruntime as ort
import numpy as np
import soundfile as sf
# Load session on CPU with 4 threads
opts = ort.SessionOptions()
opts.intra_op_num_threads = 4
session = ort.InferenceSession("kokoro_thai_q8.onnx", opts, providers=["CPUExecutionProvider"])
# Load voice style tensor (shape: [1, 256])
style = np.load("thai_som.npy")[0].astype(np.float32)
# Token IDs from FastThaiG2P frontend with BOS (0) and EOS (0)
# Example token sequence for 'ใช่ครับ'
input_ids = np.array([[0, 61, 65, 140, 172, 55, 43, 62, 173, 50, 43, 173, 0]], dtype=np.int64)
# Run inference
outputs = session.run(None, {
"input_ids": input_ids,
"ref_s": style,
"speed": np.array([1.0], dtype=np.float32)
})
audio = outputs[0].squeeze()
sf.write("output.wav", audio, 24000)
Provenance & Attribution
- Application: Text To Speech on Steam
- Base Acoustic Model: Derived from FastThaiG2P v0.3.0 (Apache-2.0, AWS Labs).
- Voice Pack (
thai_som.npy): Derived from Som TTS (CC BY). - Pronunciation Lexicon & IPA Data: Merged from Kaikki.org / Wiktionary (CC BY-SA 3.0 / GFDL) and validated phonetic dictionaries.
- License: Apache-2.0.