Qwen3-ASR-0.6B NCNN

English | 简体中文

This is a community NCNN conversion of Qwen/Qwen3-ASR-0.6B, intended for CPU/edge deployment and submodel-level inference validation.

The model was converted to NCNN FP32 with PNNX and split into five submodels: an audio convolutional frontend, an audio Transformer, token embedding, a 28-layer text decoder, and an LM head. The weights were converted from upstream checkpoint revision 5eb144179a02acc5e5ba31e748d22b0cf3e303b0.

This is not an official NCNN release from the Qwen team. The repository contains the NCNN compute subgraphs and tokenizer assets. A runtime such as ncnn-omni is still required for audio loading, Mel feature extraction, graph orchestration, prompt construction, token generation, and text post-processing.

Files

The model files total approximately 3.75 GB in decimal units. This release is not INT8/INT4 quantized and does not use FP16 weight storage.

Tokenizer and reference metadata are included alongside the model:

These assets come from the same upstream checkpoint revision as the converted weights. The original model.safetensors is intentionally not duplicated here.

Architecture

NCNN Inputs and Outputs

Blob names are defined by the corresponding .ncnn.param files. The batch dimension is omitted when passing data to NCNN; this conversion uses a fixed batch size of 1.

1. Audio Conv

The input is a 128-bin Mel chunk. Mel extraction, chunking, masking, and positional encoding are outside this subgraph.

2. Audio Transformer

This subgraph does not contain positional encoding or attention scheduling. CPU FP32 conversion validation covers boundary lengths through L=104 and longer dense cases L=196 and L=251. The Qwen3-ASR CPU/SDPA reference performs global audio attention; FlashAttention varlen windowing is a different execution policy and must not be selected silently.

3. Token Embedding

Valid token IDs are in the range [0, 151935].

4. Text Decoder

The decoder has 60 inputs and 57 outputs. Let S be the current sequence length and P the existing KV-cache length:

Use S > 1 for prefill and S = 1 for single-token decoding. The caller must generate the causal mask and RoPE cosine/sine tensors outside the graph and maintain all 56 KV-cache tensors across the 28 layers.

5. LM Head

The current model was converted and validated for one hidden state at a time. During generation, the LM head is normally applied only to the decoder hidden state of the final token.

Download

hf download songqiuyu/Qwen3-ASR-0.6B-ncnn \
  --local-dir ./Qwen3-ASR-0.6B-ncnn

You can also download the .param/.bin pairs directly from the repository's Files page. Both files of a submodel must be present.

The downloaded directory is self-contained for the current ncnn-omni Qwen3-ASR runtime. Pass the same directory as both the model and asset paths:

ncnn-omni-asr \
  --model ./Qwen3-ASR-0.6B-ncnn \
  --assets ./Qwen3-ASR-0.6B-ncnn \
  --audio ./audio.wav

NCNN Loading Example

The following example loads the token embedding model with the NCNN Python binding:

import ncnn
import numpy as np

net = ncnn.Net()
net.opt.use_fp16_packed = False
net.opt.use_fp16_storage = False
net.opt.use_fp16_arithmetic = False
net.opt.use_bf16_storage = False

assert net.load_param("embed_token.ncnn.param") == 0
assert net.load_model("embed_token.ncnn.bin") == 0

token_ids = np.asarray([0, 1, 151676], dtype=np.int32)
ex = net.create_extractor()
assert ex.input("in0", ncnn.Mat(token_ids).clone()) == 0
ret, embeddings = ex.extract("out0")
assert ret == 0

embeddings = np.asarray(embeddings)  # [3, 1024]
print(embeddings.shape)

The other submodels are loaded in the same way, but their inputs must follow the blob names, dtypes, and layouts documented above exactly.

Alignment Validation

All five submodels were tested by comparing PyTorch eager and NCNN outputs on identical inputs. The tests covered:

• zero, random, and dynamic-width boundary inputs for Audio Conv;
• multiple sequence lengths for the Audio Transformer;
• boundary and random token IDs for Token Embedding;
• zero, impulse, linear-ramp, and random inputs for the LM Head;
• prefill, single-token decode, PyTorch-generated KV caches, and NCNN-generated KV caches for the decoder.

All cases passed the predefined tolerances. Because the decoder contains 28 layers, FP32 rounding differences in attention and matrix multiplication accumulate during long prefills, so its absolute-error tolerance is higher than those of the other subgraphs. The validation also checks mean error, cosine similarity, and error relative to the reference magnitude; no graph-structure, shape, cache-ordering, or weight-conversion issue was detected.

Validation environment:

FP16, BF16, Vulkan/GPU, INT8/INT4, and the complete audio-to-text pipeline have not been validated. Run submodel-level alignment tests and end-to-end recognition evaluation before using any of these modes in production.

Limitations

• These five models are composable compute subgraphs and cannot be loaded directly with Hugging Face pipeline() or AutoModel.
• vocab.json and merges.txt are bundled and are the only asset files read directly by the current ncnn-omni runtime. The small JSON files preserve upstream reference metadata; the executable implementation of the Mel frontend and generation loop remains in the runtime.
• The caller must reproduce Audio Conv tail padding and positional encoding, then select the Audio Attention policy that matches the reference backend (global for CPU/SDPA; varlen windowing only for its corresponding FlashAttention path).
• Decoder input order, per-layer KV-cache order, and NCNN Mat dimensions must match this document exactly.

Acknowledgements and License

The original architecture and weights are from Qwen/Qwen3-ASR-0.6B and QwenLM/Qwen3-ASR. Thanks to the Qwen team for their open-source work.

This conversion follows the upstream model's declared Apache License 2.0. Use, redistribution, and modification of this repository must also comply with the upstream model license and all applicable terms.