X-ASR-zh-en

🔍 Model Card Scope

🧩 X-ASR Series

X-ASR is a series of automatic speech recognition models built with the icefall framework. The series focuses on streaming ASR and low-latency deployment, while also supporting offline recognition. The broader project roadmap, source organization, issue tracking, and bilingual documentation are maintained on the GitHub project page.

🤖 X-ASR-zh-en

X-ASR-zh-en is trained on approximately 1 million hours of open-source and collected speech data. It is designed as an offline-streaming unified transducer ASR model with the Zipformer architecture, supporting both offline decoding and true streaming decoding. The model provides multiple streaming chunk sizes: 160 ms, 480 ms, 960 ms, and 1920 ms, supports punctuation and casing, and can be deployed with sherpa-onnx.

✨ Artifact Page Notes

This repository is the model artifact page for X-ASR-zh-en.

📦 Repository Contents

Directory Layout

.
|-- README.md
|-- config.json
|-- demo/
|   `-- demo.mov
|-- applications/
|   `-- vibe-xasr/
|       |-- README.md
|       |-- download_manifest.json
|       `-- VibeXASR-1.1.2-macos-universal.dmg
|-- deployment/
|   |-- README.md
|   |-- infer_and_client/
|   |   |-- README.md
|   |   |-- sherpa_streaming_client.py
|   |   |-- sherpa_streaming_infer.py
|   |   `-- sherpa_streaming_server.py
|   `-- models/
|       |-- chunk-160ms-model/
|       |   |-- encoder-160ms.onnx
|       |   |-- decoder-160ms.onnx
|       |   |-- joiner-160ms.onnx
|       |   `-- tokens.txt
|       |-- chunk-480ms-model/
|       |-- chunk-960ms-model/
|       `-- chunk-1920ms-model/
|-- figure/
|   |-- zipformer.png
|   |-- demo-preview.png
|   `-- institutions/
`-- streaming_exp/
    `-- pretrained.pt

🧩 Model Variants

Each streaming variant contains a matched encoder, decoder, joiner, and tokens.txt. Do not mix files across model folders.

⭐ Highlights

📊 Evaluation

The following results are for the current X-ASR-zh-en release. Values are WER/CER percentages; lower is better. All results are reported with greedy search.

Note: Bold numbers indicate the best result among the listed modes for each benchmark column.

Public Benchmark Model Comparison

The following table compares representative ASR models on the same public benchmark columns. Ranks are computed by AVG across the five listed columns; lower is better. Parameter sizes are shown when provided by the source sheet.

GigaSpeechBench Vertical Domain Evaluation

The following results report GigaSpeechBench vertical-domain performance for the current X-ASR-zh-en release. Values are WER/CER percentages; lower is better. Domain abbreviations follow the GigaSpeechBench vertical-domain labels.

CH

EN

🎧 Demo

A sherpa-onnx based online demo is available here:

• https://stream-asr.sjtuxlance.com/

Demo video:

Open demo video

⬇️ Download

GitHub

Use GitHub when you want the full project repository, bilingual documentation, training references, deployment examples, and issue-tracking context.

git lfs install
git clone https://github.com/Gilgamesh-J/X-ASR.git
cd X-ASR
git lfs pull

Hugging Face

Use Hugging Face when you want the model artifact page and standard HF Hub download tooling.

hf download GilgameshWind/X-ASR-zh-en \
  --local-dir ./X-ASR-zh-en

You can also clone the Hugging Face repository with Git LFS:

git lfs install
git clone https://huggingface.co/GilgameshWind/X-ASR-zh-en
cd X-ASR-zh-en
git lfs pull

ModelScope

Use ModelScope when you prefer the ModelScope mirror or Git LFS clone from ModelScope.

git lfs install
git clone https://www.modelscope.ai/Gilgamesh-J/X-ASR-zh-en.git
cd X-ASR-zh-en
git lfs pull

🚀 Deployment

The recommended runtime is sherpa-onnx. The shortest path is to use the deployment package in this repository.

cd deployment
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Start a CPU streaming server with the 160 ms model:

python infer_and_client/sherpa_streaming_server.py \
  --host 0.0.0.0 \
  --port 8766 \
  --tokens models/chunk-160ms-model/tokens.txt \
  --encoder models/chunk-160ms-model/encoder-160ms.onnx \
  --decoder models/chunk-160ms-model/decoder-160ms.onnx \
  --joiner models/chunk-160ms-model/joiner-160ms.onnx \
  --provider cpu \
  --sample-rate 16000 \
  --feature-dim 80 \
  --num-threads 1 \
  --decoding-method greedy_search \
  --model-type zipformer2 \
  --enable-endpoint-detection 0 \
  --text-format none

Optional interactive tail-probe mode:

python infer_and_client/sherpa_streaming_server.py \
  --host 0.0.0.0 \
  --port 8766 \
  --tokens models/chunk-160ms-model/tokens.txt \
  --encoder models/chunk-160ms-model/encoder-160ms.onnx \
  --decoder models/chunk-160ms-model/decoder-160ms.onnx \
  --joiner models/chunk-160ms-model/joiner-160ms.onnx \
  --provider cpu \
  --sample-rate 16000 \
  --feature-dim 80 \
  --num-threads 1 \
  --decoding-method greedy_search \
  --model-type zipformer2 \
  --enable-endpoint-detection 0 \
  --text-format none \
  --enable-energy-tail-probe 1 \
  --low-energy-rms 0.003 \
  --speech-rms 0.010 \
  --min-speech-ms 200 \
  --min-silence-ms 500 \
  --tail-probe-ms 500 \
  --tail-probe-cooldown-ms 1000

The default mode keeps --enable-energy-tail-probe 0 and decodes only from client audio chunks. Tail-probe mode is useful for interactive voice-input demos where trailing partial results should refresh after the user pauses. Tune the RMS and silence thresholds according to microphone gain, background noise, and frontend chunking behavior.

Test it with a WAV file:

python infer_and_client/sherpa_streaming_client.py \
  --server-uri ws://127.0.0.1:8766 \
  --wav /path/to/test.wav \
  --chunk-ms 100 \
  --simulate-realtime 1

For complete runtime options, see deployment/README.md. For the script-level server/client guide and full parameter reference, see deployment/infer_and_client/README.md.

⚠️ Intended Use and Limitations

• This release is intended for Chinese-English ASR research, evaluation, demos, and deployment experiments.
• The current release focuses on streaming and offline-streaming unified recognition.
• Production latency depends on hardware, concurrency, audio chunking, endpointing, and server configuration.
• The technical report with training details, evaluation protocol, ablations, and additional analysis is coming soon.

📄 Citation

The X-ASR-zh-en technical report is coming soon. Please cite the report once it is released. For now, refer to this model card and the GitHub project page.

📜 License

This model is released under the Apache-2.0 License.

🙏 Acknowledgements

This model is trained with icefall and deployed with sherpa-onnx.

• icefall: https://github.com/k2-fsa/icefall
• sherpa-onnx: https://github.com/k2-fsa/sherpa-onnx