Whisper Large v3 Turbo — Hinglish Standardized

This model is a fine-tuned checkpoint of openai/whisper-large-v3-turbo optimized for conversational Hinglish (Hindi-English code-switching) audio, specifically targeting engineering standups, architectural reviews, and product deliberations.

🚀 Overview

• Base Architecture: openai/whisper-large-v3-turbo (32 encoder layers, 4 decoder layers, 128 mel bins, 807M params)
• Primary Domain: Bilingual Hinglish conversations, technical jargon (APIs, schemas, deployments, cloud infrastructure), and fast conversational turns.
• Dataset: 19,995 standardized speech segments transcribed and verified with standardized Hinglish orthography and phonetics.
• Format: Merged float16 PyTorch weights with Hugging Face transformers compatibility (safetensors).

📊 Training Dataset Structure & Specification

The training dataset (maintained in private storage for privacy compliance) comprises 19,995 conversational audio slices (~27.8 hours of speech) captured from real-world software engineering and product design calls.

1. Data Schema

Each record in the standardized training corpus adheres to the following JSON / JSONL structure:

{
  "audio": "data/audio/slice_000123.wav",
  "text": "so usme na PVS ka data like aapne alag se setup kiya tha na pura pipeline",
  "duration_seconds": 5.24,
  "source_meeting": "2026-09-18_16-40-38_meeting_3db1d1df"
}

2. Field Definitions

• audio (string): Relative filepath to the 16 kHz mono 16-bit WAV audio clip.
• text (string): Ground truth orthographically standardized Hinglish transcription.
• duration_seconds (float): Duration of the audio slice in seconds (range: 1.0s to 15.0s, mean: 4.8s).
• source_meeting (string): Unique identifier of the session recording session for split isolation.

3. Standardization & Orthographic Rules

To prevent Whisper token fragmentation on Romanized Hindi, the corpus was normalized using deterministic phonetic mapping:

1. Pronoun & Verb Normalization: Standardized Roman spellings for Hindi function words:
   • main (not mai / me)
   • nahin (not nahi / nai)
   • bhej dungi / kar dunga (pro-drop commitment verbs)
   • dekh leta / baat karni padegi
2. Technical Acronym Protection: Strict preservation of developer vocabulary:
   • API, SQL, UDL, JSONL, Kafka, FastConformer, MLX, Metal, PyTorch
3. Dual-Channel Audio Mixdown: All source audio pre-processed via: [0:a:0][0:a:1]amix=inputs=2:duration=longest to guarantee zero speaker channel loss.

💻 Quickstart (Transformers)

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "mayank-dubey-ai/whisper-large-v3-turbo-hinglish"

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
    use_safetensors=True
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    device=device,
)

result = pipe("audio.wav", generate_kwargs={"task": "transcribe", "language": "en"})
print(result["text"])

🛠️ Recommended Decoding Hyperparameters

In real-world conversational audio, Whisper models can exhibit repetition loops or hallucinate on background noise. To prevent decoding pathologies:

📈 Benchmark & Evaluation

Tested against live bilingual technical deliberations with reference transcriptions:

• Inference Speed: Up to 46.1x faster than real-time (RTF: 0.022x on NVIDIA GPU).
• Average Segment Latency: ~170 ms per audio slice.
• Phonetic Fidelity: Resolves common Hinglish homophones and Anglicized Hindi verb constructs (bhej dungi, kar dunga, dekh leta).

📜 Citation

@misc{dubey2026whisperhinglish,
  author = {Mayank Dubey},
  title = {Fine-Tuned Whisper-Large-v3-Turbo for Conversational Hinglish ASR},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/mayank-dubey-ai/whisper-large-v3-turbo-hinglish}}
}