Whisper-tiny Hinglish (Romanized)
A fine-tune of openai/whisper-tiny that
transcribes Hindi, English, and code-switched Hinglish Indian speech directly into
natural Roman script — the way people actually type Hinglish (kya scene hai, not क्या सीन है
and not a stiff academic transliteration).
- 39 M parameters, 4 encoder + 4 decoder layers, 80-mel, 16 kHz — runs on CPU / edge.
- Output is always Roman, even for pure-Hindi speech. No Devanagari, no separate transliteration step at inference time.
- Built for on-device dictation for Indian users, but released here as a standalone, general-purpose model for the community.
This is a small model. It is fast and convenient, not state-of-the-art. See Limitations.
Example outputs
The model emits Roman text directly from audio. Target style (illustrative):
| Speech (reference) | Model output style |
|---|---|
| "…with my daughter और जो मेरी daughter है उसको बहुत पसंद है diwali…" | …with my daughter aur jo meri daughter hai usko bahut pasand hai diwali… |
| "वीर बड़े भाई known for his curiosity and दीया छोटी बहन…" | veer bade bhai known for his curiosity and deeya chhoti bahan… |
| "आपका hometown किस city में है…" | aapka hometown kis city mein hai… |
| "दिवाली मनाते हैं साथ में पटाखे जलाते हैं" | diwali manate hain saath mein patakhe jalate hain |
English spans are kept verbatim; Hindi spans come out in natural Roman spelling.
Usage
from transformers import pipeline
asr = pipeline("automatic-speech-recognition",
model="<your-username>/whisper-tiny-hinglish",
chunk_length_s=30) # for clips longer than 30 s
print(asr("clip.wav")["text"])
Or with raw 16 kHz mono audio:
import soundfile as sf
audio, sr = sf.read("clip.wav") # resample to 16 kHz mono if needed
print(asr({"raw": audio, "sampling_rate": 16000})["text"])
The model does not require a forced language token — it was trained to emit Roman for every input. No post-processing / transliteration is needed.
Training data
Fine-tuned on Indian-accent, read- and prompted-speech corpora, chosen for their heavy Hindi–English code-switching:
| Source | Content |
|---|---|
| HiACC (adult + children) | Indian-accented Hindi/English/Hinglish read & conversational speech |
| MUCS (Hindi–English) | Code-switched Hindi–English utterances (Kaldi-segmented) |
| Common Voice (Hindi / Indic) | Crowdsourced Hindi read speech |
Utterances were filtered to ~0.5–20 s. Original transcripts are mixed Devanagari + Latin.
Romanization of the targets (the key step)
The goal is Roman output. So every training target was converted to natural Roman before fine-tuning:
- Devanagari spans → romanized with LIPI, a
compact CTC Devanagari→Roman model that produces typed-Hinglish spellings
(
क्या→kya,है→hai,जो→jo), not academic IAST. - English (Latin) spans in the transcripts were left unchanged.
The model then learns audio → Roman end-to-end, so LIPI is only a data-prep tool and is not needed at inference.
Strengths
- Natural romanized Hinglish output, ready to paste — no transliteration step downstream.
- Handles code-switching within a sentence (Hindi ↔ English) in one pass.
- Tiny & fast: real-time on CPU, sub-second on a modest GPU; ~150 MB fp32 (quantizes well).
- Robust to Indian accents on short, clear, conversational/dictation-style audio.
Limitations
- Small-model accuracy. Expect errors on noisy audio, fast speech, strong accents, overlapping speakers, or domain jargon.
- English spelling drift. Uncommon English words can come out phonetically
(
crackers→crackers/crakers), since the training signal is Indian-accented and Hindi-weighted. - Romanization is phonetic, not canonical. Spellings follow how Hindi is typed and can vary
(
hai/hain,ki/kii). There is no single "correct" Roman spelling. - Short-form. Trained on ≤ ~20 s utterances; use
chunk_length_s=30for longer audio, quality on very long recordings is untested. - No reliable punctuation/casing. Output is lightly punctuated at best.
- Language scope. Hindi + English + Hinglish (Indian). Other Indic languages are out of scope.
- Domain. Read/prompted Indian speech; far-field, telephony, and heavy background noise are under-represented.
License
The model weights are released under the MIT License, the same as the base model openai/whisper-tiny. You can use, modify, and redistribute the model, including in commercial and free apps.
Training data
The model was fine-tuned on:
- HiACC (Hinglish Adult & Children Code-switched Corpus): CC BY 4.0, https://doi.org/10.5281/zenodo.15551669
- MUCS 2021 Hindi-English (OpenSLR SLR104): CC BY-SA 4.0, https://openslr.org/104/
- Common Voice (English): CC0
No dataset audio or transcripts are distributed with this model. MUCS is CC BY-SA 4.0. Whether ShareAlike terms carry over to model weights trained on such data is legally unsettled. If this matters for your use case, please review those terms yourself.
Citation
@misc{whisper-tiny-hinglish,
title = {Whisper-tiny Hinglish (Romanized)},
author = {Abhishek Gautam},
year = {2026},
note = {Fine-tuned openai/whisper-tiny for romanized Hindi/English/Hinglish ASR;
targets romanized with LIPI (https://github.com/ABHISHEKgauti25/lipi)}
}