Bro-ASR-1.7B
Bro-ASR-1.7B is a merged, parameter-efficiently fine-tuned derivative of
Qwen/Qwen3-ASR-1.7B-hf for Japanese noisy-recording/device-domain speech
recognition. The repository contains full merged model weights; it does not
require a separate LoRA adapter at inference time.
Intended use
- Offline Japanese ASR for noisy conversational recordings.
- Research and evaluation of device/noise-domain ASR adaptation.
It is not a safety-critical transcription system. Validate performance on your own audio domain before deployment.
Training
The model was produced by merging a LoRA adapter trained with the frozen
lowrank_loraplus8_coverage recipe:
- Base model:
Qwen/Qwen3-ASR-1.7B-hf - Training pool: 3,000 Delivery_JP Japanese audio/text segments
- LoRA: rank 8, alpha 16, dropout 0.1, applied to attention and MLP projections
- LoRA+: base learning rate
8e-6, B-matrix ratio8x - Sampling: shuffle without replacement; 500 micro-steps, gradient accumulation 2
The Delivery_JP audio and transcripts are not included in this repository.
Dataset and evaluation
Evaluation uses the 20,619-segment Delivery_JP Japanese manifest. The shared
3,000-segment training/development pool is excluded before scoring, leaving a
strictly unseen evaluation set of 17,619 segments. Both models were run on
the same ordered sample_id list with identical normalized ground truth; each
side produced exactly 17,619 predictions with no duplicate IDs.
Decoding uses a Japanese language hint, beam size 5, and
max_new_tokens=256. CER is character-level Levenshtein distance after the
same whitespace normalization for both models. Macro-CER is the unweighted
mean of per-segment CER.
| Model | Evaluated segments | Micro-CER | Macro-CER | Character errors | Exact-match rate |
|---|---|---|---|---|---|
| Base Qwen3-ASR-1.7B | 17,619 | 0.34295 | 0.41445 | 102,563 | 14.995% |
| Bro-ASR-1.7B | 17,619 | 0.33763 | 0.39566 | 100,970 | 18.208% |
Compared with the base model, Bro-ASR-1.7B reduces Micro-CER by 0.533 percentage points and character errors by 1,593. On paired per-segment comparison, Bro-ASR-1.7B is better / unchanged / worse on 5,681 / 7,775 / 4,163 segments, respectively.
Fixed-set audit score comparisons
The following four models were evaluated on the identical frozen 1,000
sample_id values with identical normalized ground truth and exact fractional
audio time boundaries. This fixed-set audit is separate from the 17,619-segment
strict-unseen evaluation above.
| Fixed audit subset | Model | Segments | Micro-CER | Macro-CER | Character errors | Exact-match rate | Decoding |
|---|---|---|---|---|---|---|---|
| Delivery_JP frozen 1,000 | Bro-ASR-1.7B | 1,000 | 0.47119 | 0.49852 | 7,581 | 14.4% | Japanese, beam 5 |
| Delivery_JP frozen 1,000 | Qwen3-ASR-1.7B baseline | 1,000 | 0.48574 | 0.54712 | 7,815 | 10.3% | Japanese, beam 5 |
| Delivery_JP frozen 1,000 | Cohere Transcribe 03-2026 | 1,000 | 0.56871 | 0.63912 | 9,150 | 7.1% | Official model.transcribe, ja, punctuation, greedy |
| Delivery_JP frozen 1,000 | Whisper large-v3-turbo (untrained) | 1,000 | 1.53969 | 3.86799 | 24,772 | 0.7% | Historical frozen TTA: speed factors 1.0/0.95/1.05, Japanese, beam 5 |
On this 1,000-segment audit, Bro-ASR-1.7B reduces Micro-CER by 1.454 percentage points and character errors by 234 relative to the paired Qwen baseline. The Cohere run used the official native transcription entry point with EOS handling; Qwen and Bro-ASR both used Japanese beam size 5.
The Whisper row is the untrained base model, not the historical Plan B
fine-tuned checkpoint. It was reproduced with the frozen
best_solution_snapshot.py TTA decoder and an asserted zero LoRA B-matrix.
Its result is substantially weaker on this audit; the decode policy is
reported separately because it uses three TTA speed factors while the Qwen
runs use beam-5 decoding.
Quick start
Qwen3-ASR is not included in older stable Transformers releases. Install the current Transformers source first, as recommended by the upstream Qwen3-ASR model card:
pip install "git+https://github.com/huggingface/transformers"
pip install torch soundfile
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "Junlaii/Bro-ASR-1.7B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()
inputs = processor.apply_transcription_request(
audio="sample.wav", language="ja"
).to(model.device, model.dtype)
with torch.inference_mode():
output = model.generate(**inputs, num_beams=5, max_new_tokens=256)
text = processor.decode(
output[:, inputs["input_ids"].shape[1]:], return_format="transcription_only"
)[0]
print(text)
Limitations
- The current adaptation is specifically motivated by Japanese noisy-recording conditions; it may regress on other languages or acoustic domains.
- The full strict-unseen Delivery_JP evaluation has completed on 17,619 segments under the protocol described above.
- Decoding settings affect results. The final comparison uses Japanese,
beam size 5, and
max_new_tokens=256.
License and attribution
Bro-ASR-1.7B follows the Apache-2.0 license indicated by its base checkpoint. Please cite and follow the upstream Qwen3-ASR model terms when using this model.