GPA: One Model for Speech Recognition, Text-to-Speech, and Voice Conversion

TL;DR GPA incorporates three speech tasks into one single model and this repo includes codes of training, fine-tuning and effecient deployment of GPA.

🆕 GPA-TTS Updates

GPA-TTS is a standalone, ultra-efficient TTS runtime distilled from GPA, designed for edge deployment.

• INT8/INT4 quantized: among the smallest open-source TTS runtimes • Runtime-selectable decoder: INT8 / FP16 / FP32 (quality vs. efficiency trade-off)
• Zero-shot voice cloning from short reference audio
• Fully local: no external LLM required
• Production-ready REST API with voice management

Built for developers deploying voice applications on resource-constrained devices.

📖 Abstract

GPA stands for General Purpose Audio.

In academia, a student’s GPA (Grade Point Average) serves as a unified metric that reflects performance across diverse subjects—ranging from Calculus and Philosophy to Gym class.

Similarly, our GPA model unifies the three major pillars of audio tasks—Text-to-Speech (TTS), Automatic Speech Recognition (ASR), and Voice Conversion (VC)—into a single auto-regreesive transformer.

• Our open-source content includes support for multiple frameworks and provides production-ready code suitable for cloud deployment.
• we include concise inference examples and training pipelines for research purpose.
• The released 0.3B model is also perfect for edge devices and edge deployment is to be released.

🔍 Model Overview

⚡ Model Performance

The following results are obtained by benchmarking services instantiated via the official deployment scripts, reflecting end-to-end performance in realistic serving scenarios rather than offline inference.

Among currently available open-source systems, our model is one of the few that natively supports both concurrent and streaming inference, while achieving performance comparable to the first tier of existing approaches.

💡Note

• TTFC: Time To First Chunk (TTS)
• TTFT: Time To First Token (ASR)
• RTF: Real-Time Factor (audio duration / synthesis time)

TTS Streaming Benchmark (Latency & Throughput)

ASR Streaming Benchmark

📊 Evaluation Metric Results

TTS Evaluation Table

ASR Evaluation Table

Note: ASR results on Librispeech and Aishell-1. WER (%) is reported for Librispeech, and CER (%) is reported for Aishell-1.

🙏 Acknowledgements

We borrowed a lot of code from the following excellent projects:

• Spark-TTS
• GLM-4-Voice
• Emilia
• FlashTTS
• Qwen

🔗 Citation

If you find GPA useful for your research or projects, please cite us:

@misc{cai2026unifyingspeechrecognitionsynthesis,
      title={Unifying Speech Recognition, Synthesis and Conversion with Autoregressive Transformers}, 
      author={Runyuan Cai and Yu Lin and Yiming Wang and Chunlin Fu and Xiaodong Zeng},
      year={2026},
      eprint={2601.10770},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2601.10770}, 
}