Bengali ASR with punctuation restoration

Bengali speech in, punctuated Bengali text out. A whisper-medium fine-tune — the ASR half of the 1st-place solution of the Bengali.AI Speech Recognition Kaggle competition (0.312 WER, public leaderboard). Loads directly with standard transformers tooling, no custom code required.

Attribution. Trained by tugstugi (Erdene-Ochir Tuguldur), team Chimege; redistributed here under Apache-2.0. This repo only adds packaging (safetensors conversion, a repaired generation config). Please cite the original author — see Citation.

Want punctuated output? Pair this with asr-punctuation-restore-bn (same competition solution's MuRIL punctuation heads, published standalone — works on any ASR's output) via the asr-punct-restore package, used below.

Quick start

Stage 1 — ASR:

from transformers import pipeline

asr = pipeline("automatic-speech-recognition",
               model="SayedShaun/bengali-whisper-medium",
               chunk_length_s=20.1)  # >20s: competition audio is long-form

result = asr("clip.wav", generate_kwargs={"language": "bn", "task": "transcribe",
                                          "num_beams": 4, "max_length": 260})
raw_text = result["text"]  # no punctuation yet

Apply NFC normalization before comparing or storing output — the model emits precomposed Bengali letters (য় ড় ঢ়) that Bengali corpora write as base+nukta; they render identically but compare unequal (cost 18% WER on one otherwise-perfect sample):

import unicodedata
raw_text = unicodedata.normalize("NFC", raw_text)

Stage 2 — punctuation:

pip install git+https://github.com/sayedshaun/asr-punct-restore.git

from asr_punct_restore import PunctuationRestorer

restorer = PunctuationRestorer("SayedShaun/asr-punctuation-restore-bn",
                               layers=(12,))  # most accurate single head
punctuated = restorer(raw_text)

In production

• Pin a revision so a later push here can't change what you serve.
• Warm up both stages once at startup — first real request shouldn't pay kernel-autotune / model-load cost.
• One instance per worker, reused across requests — not thread-safe for concurrent calls.

Limitations

• Bengali only.
• Hallucinates fluent text on silence/noise like all Whisper models — gate with VAD.
• Punctuation covers only ।, ,, ? — no prosody, no exclamation points.
• Public/private WER gap (0.312 / 0.372) suggests YouTube pseudo-labeling fit the public test domain somewhat.

Citation

Please cite the original author, not this packaging:

@misc{tuguldur2023bengaliasr,
  author = {Tuguldur, Erdene-Ochir},
  title  = {1st place solution, Bengali.AI Speech Recognition},
  year   = {2023},
  howpublished = {\url{https://www.kaggle.com/competitions/bengaliai-speech/writeups/chimege-1st-place-solution}}
}

• Writeup: https://www.kaggle.com/competitions/bengaliai-speech/writeups/chimege-1st-place-solution
• Original weights: tugstugi/bengali-ai-asr-submission on Kaggle, mirrored at bengaliAI/tugstugi_bengaliai-asr_whisper-medium
• Competition: https://www.kaggle.com/competitions/bengaliai-speech

License

Apache-2.0, following the original upload; the Kaggle submission bundle is CC0.