Ppaso-TTS — Korean lightweight TTS (RK3576 NPU friendly)

한국어: README_ko.md

Ppaso (빠소) = short for ppareunsori (빠른소리, "fast voice" in Korean). Real-time Korean TTS for edge NPU devices.

What it is for

When pairing an LLM with TTS on an edge device (e.g. NanoPi R76S) for a Korean voice assistant, TTS inference often becomes the latency bottleneck. Ppaso-TTS is a lightweight TTS designed to remove that bottleneck by running on the NPU.

Designed for speed and small footprint over naturalness. Single voice, Korean only.

Highlights

• Korean only, single female voice. Speed and lightweightness prioritized over quality (robotic timbre included).
• ~20× real-time on NPU — ~305 ms for a 5.94-second utterance on RK3576 (RTF 0.052).
• 21 MB on-device model (RKNN) — fits comfortably on RAM-constrained edge devices.
• Two backends: RKNN NPU (RK3576 / RK3588) / ONNX CPU (anywhere).
• Any-length input — built-in chunking handles long paragraphs without truncation.
• Streaming mode — feed LLM tokens as they arrive; emit wav as soon as a sentence completes (voice assistant friendly).
• 🆕 Lipsync animation — companion module ppaso_lipsync generates 2D mouth + 1D eye-blink animation curves directly from TTS output. One curve drives two render routes: 2D sprite (atlas) and 3D blendshape (VRM). Pure numpy, edge-friendly. See demo video below.
• Custom vocabulary focus — training corpus is self-synthesized via a CosyVoice teacher (not an external dataset). Naturalness is sacrificed, but the data distribution for domain-specific vocabulary (industrial safety, technical terms, etc.) can be controlled.

What's New (v8 — 2026-05-30)

v8 is a from-scratch retrain on a refined training corpus. No architecture / config change.

• Cleaner pronunciation for liaison, aspiration and morpheme-boundary patterns (맑→말금, 흙이→흘기, 좋고 / 좋다 aspiration). The G2P lexicon was rebuilt with explicit phonology rules; v8 is the first acoustic model trained on the new label distribution.
• Stabilized word endings — short utterances (e.g. 이 약 좋다.) occasionally lost their final syllable in the teacher-synthesized corpus. v8 quarantines 27 such samples and regenerates 25 of them with a postfix-trim workaround, removing a source of trailing instability.
• Audio samples (samples/*.wav) and lipsync mp4 (samples/*.mp4) re-rendered with v8.

How it works

Korean text  →  IPA phones  →  mel  →  wav (22.05 kHz mono)

Internally a G2P (mecab-ko + MFA lexicon) + 4 NN modules (RKNN/ONNX) + iSTFT (CPU). For users it's a single function call.

Performance (RK3576, 5.94 s single utterance)

End-to-end measurements (warmup + median, single chunk, full pipeline — g2p, NN inference, iSTFT all included):

NPU is ~1.6× faster than CPU on the same device.

Audio Samples

samples/ (RKNN NPU output):

Lipsync Animation (optional)

🆕 Companion pure-numpy module ppaso_lipsync — turns TTS output into per-frame 2D mouth + 1D eye-blink animation curves. One curve drives two render routes: atlas (2D sprite) and blendshape (3D VRM morph). TTS overhead < 1 ms (wav byte-identical).

from ppaso_tts import PpasoTTS
from ppaso_lipsync import generate_blink_timeline

tts = PpasoTTS('./', backend='onnx')
wav, lipsync = tts.synthesize_with_lipsync('안녕하세요.', fps=30)
blink = generate_blink_timeline(duration=lipsync.duration, fps=30, seed=42)
# lipsync.xy_timeline (n_frames, 2)  +  blink.state_timeline (n_frames,)

Render to mp4: python example/07_avatar_video.py (atlas, 2D) · python example/08_vrm_video.py (blendshape, 3D VRM).

Full design / API / performance → README_lipsync.md. Blendshape mapping detail → README_bs2d.md.

Quick Start

Install

pip install numpy soundfile python-mecab-ko onnxruntime
# For RK3576 / RK3588 NPU:
pip install rknn-toolkit-lite2
# For 07_avatar_video.py (mp4 render demo):
pip install Pillow          # + system ffmpeg (apt install ffmpeg)

Hello World

from ppaso_tts import PpasoTTS
import soundfile as sf

tts = PpasoTTS('./', backend='onnx')         # ONNX (CPU/CUDA, anywhere)
# tts = PpasoTTS('./', backend='rknn')       # RKNN (RK3576 / RK3588 NPU)

wav = tts.synthesize("안녕하세요. 자기소개 부탁드립니다.")
sf.write('hello.wav', wav, 22050)

Long input — auto-chunked:

wav = tts.synthesize(
    "안녕하세요. 오늘 날씨가 정말 좋네요. 산책 어떠세요? "
    "긴 문장도 끝부분 잘림 없이 자동적으로 합성됩니다."
)
sf.write('long.wav', wav, 22050)

Custom pronunciations (user dict)

Pass a user_dict mapping any input form → Korean spelling. Replacement is applied automatically before synthesis:

tts = PpasoTTS('./', backend='onnx', user_dict={
    "pok3r":   "포커",
    "GPT":     "지피티",
    "ChatGPT": "챗 지피티",
})
wav = tts.synthesize("pok3r and GPT are both spoken naturally as Korean.")

Streaming (voice assistant)

LLM token stream → wav emitted on each sentence completion:

from ppaso_tts import PpasoTTS, StreamingTTS

tts = PpasoTTS('./', backend='rknn')
stream = StreamingTTS(tts)

for token in llm_token_stream():        # hook your LLM token stream
    for wav in stream.feed(token):      # emit on strong punct (.!?)
        play(wav)                       # play immediately
for wav in stream.flush():              # end of dialog — drain buffer
    play(wav)

More examples

Examples in example/:

Limitations

• Korean only — other languages will be mispronounced.
• Single speaker — no voice cloning.
• Mild robotic timbre persists (capacity / speed trade-off).
• Loanwords / English transliteration may sound unnatural.
• Long-input chunking — arbitrary length is handled by built-in chunking, but prosody (intonation / pace) consistency across chunk boundaries is somewhat weak.
• Rare pronunciations — words missing from the bundled lexicon (~25 K entries) fall back to character-level phones, which can sound off.
• No personal identity — does not represent any real person; do not use for impersonation.

Intended / Out-of-Scope Use

Intended:

• Edge AI voice assistants (Korean speakers)
• IoT voice notifications (announcements, alarms)
• Accessibility tooling (screen readers)

Prohibited:

• ❌ Voice impersonation of specific individuals
• ❌ Deepfake / fraudulent audio generation
• ❌ Standalone use in critical systems (medical, legal, emergency) without human review
• ❌ Spam / fraud automation

Repository Layout

.
├── README.md / README_ko.md     # This file (English / Korean)
├── LICENSE                      # Apache 2.0
├── config.json                  # Model metadata
│
├── onnx/                        # CPU/CUDA inference (37 MB)
├── rknn/                        # RK3576 / RK3588 NPU only (FP16, 21 MB)
├── runtime/                     # G2P + lexicon
├── example/                     # 8 example scripts + ppaso_tts.py class
│   ├── ppaso_lipsync/           #   ↳ 9-viseme lipsync + eye blink + bs2d (numpy)
│   ├── ppaso_vrm/               #   ↳ VRM 3D blendshape renderer (route 2)
│   ├── avatar/                  #   ↳ sample 2D avatar for 07
│   └── avatar_vrm/              #   ↳ sample VRM avatar for 08 (CC0)
└── samples/                     # Synthesized waveforms + lipsync mp4

License

Apache License 2.0 — free for commercial use, includes patent grant, requires preservation of NOTICE / attribution.

Attribution

External resources used in training this model:

• Fun-CosyVoice 3.0 (Apache 2.0) — used as a teacher to synthesize the Korean training corpus. The teacher weights are not redistributed in this repo.
• Vocos (MIT) — the ConvNeXt-1D + iSTFT vocoder design is inspired by Vocos. Our vocoder is trained from scratch and does not include any Vocos weights.
• MFA korean_mfa.dict (MIT, Eleanor Chodroff)
• mecab-ko (BSD)

Bundled sample avatar assets (example/, not used in training):

• example/avatar/avatar.png — AI illustration generated with an Illustrious XL-based model (Fair AI Public License 1.0).
• example/avatar_vrm/AvatarSample_F.vrm — VRoid Studio sample model AvatarSample_F (formerly Vita), CC0 (public-domain dedication) by the VRoid Project.

Acknowledgments

• This project was developed with support from i-nx.com.
• FunAudioLLM team — Fun-CosyVoice 3.0 and its permissive license.
• Vocos authors — ConvNeXt-1D vocoder architecture.
• Rockchip — RK3576 NPU documentation and RKNN toolkit.
• Anthropic Claude — architecture exploration, training pipeline, and model card documentation.

Citation

@misc{ppaso-tts,
  author = {akamotaco},
  title  = {Ppaso-TTS: Korean edge-optimized TTS for RK3576 NPU},
  year   = {2026},
  url    = {https://huggingface.co/akamotaco/ppaso-tts-v1},
  note   = {Developed with support from i-nx.com, and AI assistance from Anthropic Claude.}
}

Teacher model:

@article{du2025cosyvoice,
  title   = {CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training},
  author  = {Du, Zhihao and others},
  journal = {arXiv preprint arXiv:2505.17589},
  year    = {2025}
}

Vocoder design:

@inproceedings{siuzdak2024vocos,
  title     = {Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis},
  author    = {Siuzdak, Hubert},
  booktitle = {ICLR},
  year      = {2024}
}

한국어 버전: README_ko.md