Nemotron-3.5-ASR-Streaming-0.6B — zh-TW fine-tuned (v3)
Fine-tune of nvidia/nemotron-3.5-asr-streaming-0.6b specialised for Taiwan Mandarin. v3 improves zh-TW by a further 18% over v2 (Common Voice zh-TW CER 12.03 → 9.81), for small regressions on a few other languages — see the honest table below.
Results (transformers runtime, greedy, script-normalised via OpenCC)
| eval | base* | v2 | v3 (this) |
|---|---|---|---|
| Common Voice zh-TW (n=120, CER) | 38.43 | 12.03 | 9.81 ✅ |
| Taiwan-Tongues test (n=100, CER) | 30.67 | — | 14.85 |
| English LibriSpeech (n=100, WER) | 2.71 | 2.58 | 2.79 |
| Arabic FLEURS (WER) | 14.59 | 14.80 | 14.48 ✅ |
| Japanese FLEURS (CER) | 16.89 | 16.48 | 16.65 ✅ |
| Korean FLEURS (CER) | 9.60 | 8.80 | 9.42 ✅ |
| German FLEURS (WER) | 11.67 | 10.92 | 11.42 ✅ |
| Hindi FLEURS (WER) | 8.08 | 7.91 | 8.80 |
| Spanish FLEURS (WER) | 5.56 | 6.83 | 6.75 |
| French FLEURS (WER) | 13.90 | 15.45 | 16.15 ⚠️ |
* base = same model with the zh-TW prompt column warm-started from zh-CN (the untouched base zh-TW slot is dead, ~100% CER). Choose v2 if you need French/Hindi/English at their best; choose v3 for zh-TW. French is the one number that exceeds NVIDIA's published FLEURS figure (15.93).
Output is Simplified Chinese — convert with OpenCC
The 13,087-token tokenizer cannot represent many common Traditional characters (點 兒 區 說 麼 嗎 灣 黨 體 產 …; 6.3% of Traditional chars fail a round-trip vs 0.1% for Simplified). Training targets were Simplified on purpose. For Traditional:
from opencc import OpenCC
text = OpenCC("s2t").convert(text)
Any of language="auto" / "zh-CN" / "zh-TW" works (within ~0.7 CER of each other).
Training
NVIDIA's NeMo recipe (speech_to_text_finetune.py,
fastconformer_transducer_bpe_streaming_prompt.yaml), 2×RTX 5090 DDP, bf16,
AdamW lr=1e-5 (no Noam — the config's lr:2.0 is a pretraining setting).
Phase 1 (47.5 h): 12,077 Common Voice zh-TW + 5,846 IVOD 立法院 segments kept
only where two independent teachers agreed (X-ASR and Whisper-medium disagreed
by >20% on 41% of segments — those were discarded) + 4,516 LibriSpeech English +
600 ASCEND zh-CN replay. prompt_mode: unified.
Phase 2 (repair, lr 4e-6, 3 epochs): English-heavy mix to undo the English
drift — recovered en 2.92 → 2.79 with zero zh-TW loss.
Limitations
- A dedicated zh-en model is still better at Chinese: an X-ASR zipformer scores 6.66 CER (Common Voice zh-TW) and 6.13 (Taiwan-Tongues test) on these same clips. This model's value is covering 25 languages, not beating a zh-en specialist.
- Long-form literary Chinese is weak (FormosaSpeech 23.4 CER) — classical proper nouns and vocabulary are out of domain.
- ASR only: no diarization, no timestamps.
Provenance
Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License).
Common Voice zh-TW (CC0) via JacobLinCool/common_voice_16_1_zh_TW_clean;
立法院 IVOD open data (CC-BY-4.0) pseudo-labelled and teacher-filtered;
LibriSpeech (CC-BY-4.0); ASCEND.