Qwen3-ASR-1.7B - Renesas X5H
Introduction
This repository contains Qwen3-ASR-1.7B model, which support language identification and speech recognition for 30 languages and 22 Chinese dialects, optimized for Renesas X5H platform for automatic speech recognition inference.
- Model Architecture:
-
Model Summary:
Audio Encoder:
Parameter Value NUM_LAYERS 24 HIDDEN_SIZE 1024 FFN_DIM 4096 NUM_HEADS 16 HEAD_DIM 64 MEL_BINS 128 OUTPUT_DIM 2048 Text Decoder:
Parameter Value NUM_LAYERS 28 HIDDEN_SIZE 2048 FFN_DIM 6144 NUM_HEADS 16 NUM_KV_HEADS 8 HEAD_DIM 128 GROUP_SIZE 2 VOCAB_SIZE 151936 RMS_NORM_EPS 1e-6 ROPE_THETA 1000000.0 -
Source Model: Qwen/Qwen3-ASR-1.7B
Performance
The following performance metrics were measured with a 15-second English audio sample.
| Model | Precision | Device | Offloading | Prefill Rate (tokens/sec) Batch size=1 | Decode Rate (tokens/sec) |
|---|---|---|---|---|---|
| Qwen3-ASR-1.7B | W4A16 | X5H - Single Cluster NPX | AudioEncode [NPX] LLM Decoder [NPX] | 20.84 | 20.71 |
| Qwen3-ASR-1.7B | FP16 | X5H - Single Cluster NPX | AudioEncode [NPX] LLM Decoder [NPX] | 12.28 | 12.23 |
Prerequisites
To run model, you need:
- Renesas X5H Board with SDK v4.34.0
- Hugging Face CLI: For downloading the model and installer.
Deployment
Qwen3-ASR-1.7B (W4A16)
- Download the installer qwen3-asr-w4a16-runner-0.1.0-Linux.sh from Files and version tab under w4a16/binaries/rcar-x5hv1/xOS-v4.34/ folder.
- Copy the installer to the X5H board and run the installer.
bash ./qwen3-asr-w4a16-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license - Download GGUF files Qwen3-ASR-1.7B-FP16.gguf and mmproj-Qwen3-ASR-1.7b-FP16.gguf from Files and version tab under fp16 folder and copy to the installed directory on the X5H board.
- Download the W4A16 coefficients from w4a16/coefficients/ and copy to the installed directory.
- Expected directory structure on the X5H board.
qwen3-asr-w4a16-runner ├── Qwen3-ASR-1.7B-FP16.gguf ├── mmproj-Qwen3-ASR-1.7b-FP16.gguf ├── firmwares ├── kernel_modules ├── qwen3-asr-w4a16 ├── qwen3-asr-w4a16-runner ├── scripts ├── setup_npu.sh └── test_audio └── sample_en.wav
Inference - Qwen3-ASR-1.7B (W4A16)
bash ./setup_npu.sh
./qwen3-asr-w4a16-runner -m Qwen3-ASR-1.7B-FP16.gguf -v mmproj-Qwen3-ASR-1.7b-FP16.gguf -w test_audio/sample_en.wav -g ./ -s
Qwen3-ASR-1.7B (FP16)
- Download the installer qwen3-asr-runner-0.1.0-Linux.sh from Files and version tab under fp16/binaries/rcar-x5hv1/xOS-v4.34/ folder.
- Copy the installer to the X5H board and run the installer.
bash ./qwen3-asr-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license - Download GGUF files Qwen3-ASR-1.7B-FP16.gguf and mmproj-Qwen3-ASR-1.7b-FP16.gguf from Files and version tab under fp16 folder and copy to the installed directory on the X5H board.
- Expected directory structure on the X5H board.
qwen3-asr-runner ├── Qwen3-ASR-1.7B-FP16.gguf ├── mmproj-Qwen3-ASR-1.7b-FP16.gguf ├── firmwares ├── kernel_modules ├── qwen3-asr ├── qwen3-asr-runner ├── scripts ├── setup_npu.sh └── test_audio └── sample_en.wav
Inference - Qwen3-ASR-1.7B (FP16)
bash ./setup_npu.sh
./qwen3-asr-runner -m Qwen3-ASR-1.7B-FP16.gguf -v mmproj-Qwen3-ASR-1.7b-FP16.gguf -w test_audio/sample_en.wav -g ./ -s