Tara: Frontier Hindi Transcription Model

Watch the launch video here. Try via API here.

Tara is a frontier automatic speech-recognition model for Hindi and mixed-code (Hinglish) transcription. On the AI4Bharat Vistaar Hindi benchmark suite it achieves state-of-the-art aggregate accuracy, outperforming leading commercial Hindi ASR systems on the 7-benchmark Vistaar mean, while natively handling Hindi–English code-switched speech through a dedicated mixed-code mode that renders English words in Latin script and Hindi in Devanagari, the way real Hinglish is written.

Highlights

• State-of-the-art Vistaar Hindi aggregate: 12.06 WER mean over the 7 Vistaar sets, ahead of Sarvam Saaras-v3 (12.32), with wins on Kathbath, GramVaani, IndicTTS and CommonVoice-hi.
• Native code-switching: 8.37 WER on Code-Switch FLEURS (CS-FLEURS) Hindi–English read code-switch via Tara's mixed-code mode, competitive with the best commercial systems.
• Robust across domains: read speech, noisy speech, telephony (GramVaani 21.03 vs Sarvam 23.00), spontaneous conversation (IndicVoices), and accented adult/child speech (HiACC).
• Bilingual: retains strong English (6.68 WER CommonVoice-en, 4.55 FLEURS-en).
• Standard tooling: loads with 🤗 Transformers exactly like openai/whisper-large-v3.

Usage

import librosa
import torch
from transformers import WhisperProcessor, WhisperForConditionalGeneration

repo = "Trelis/tara"
processor = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(
    repo, torch_dtype=torch.bfloat16).to("cuda")

tk = processor.tokenizer
hi, en, mc = (tk.convert_tokens_to_ids(t) for t in ("<|hi|>", "<|en|>", "<|mixedcode|>"))
trn, nts = (tk.convert_tokens_to_ids(t) for t in ("<|transcribe|>", "<|notimestamps|>"))

audio_16k, _ = librosa.load("clip.wav", sr=16000, mono=True)
feats = processor(audio_16k, sampling_rate=16000,
                  return_tensors="pt").input_features.to("cuda", torch.bfloat16)

# Example 1: pure Hindi
out = model.generate(input_features=feats,
                     forced_decoder_ids=[(1, hi), (2, trn), (3, nts)],
                     max_new_tokens=444)
print(tk.decode(out[0], skip_special_tokens=True))

# Example 2: Hindi-English mixed-code, inject <|mixedcode|> right after the language token.
# Language auto-detection also works: generate one step unforced and the FIRST generated
# token is the language token; then inject <|mixedcode|> after it and continue.
out = model.generate(input_features=feats,
                     forced_decoder_ids=[(1, hi), (2, mc), (3, trn), (4, nts)],
                     max_new_tokens=444)
print(tk.decode(out[0], skip_special_tokens=True))

The mixed-code mode (the <|mixedcode|> prefix above) conditions generation only: on pure-Hindi audio it neither degrades accuracy nor forces transliteration; on mixed-code audio it renders English words in Latin script.

Evaluation

Evaluation code, the exact text normalizer, and Tara's per-utterance predictions for every benchmark below are published at TrelisResearch/tara, so all numbers can be reproduced or re-scored under alternative normalizers.

Protocol. All numbers are corpus WER after light text normalization* (Unicode NFC plus punctuation removal; nukta and all vowel and nasal marks preserved). All systems are scored on clips ≤ 30 s with identical references. Commercial-system results are measured by us under the same protocol; they are not vendor-reported figures.

Vistaar Hindi benchmark (WER ↓)

IndicVoices-500 is a 500-sample spontaneous-speech control from the IndicVoices validation split; it is not part of the Vistaar mean.

Code-switching (Hinglish) benchmarks (WER ↓)

Tara and Sarvam are measured in their code-mixed modes.

English (WER ↓)

Scored with the standard Whisper English normalizer.

* Normalization: unicodedata.normalize("NFC"), lowercasing, then removal of punctuation and symbols (। , . ? ! " : ; - – — “ ” ( ) [ ] < > / ~ % ₹ $ …), invisible formatting characters (zero-width joiner/space) and the Unicode replacement character; apostrophes are kept. The ≤30 s rule excludes 2 clips on GramVaani, 2 on IndicTTS and 1 on FLEURS-hi; no other set has any. Measurement error is small: re-runs across hardware and precision agree to within 0.1 WER. There is also slight noise in the reference labels (for example inconsistent nukta spelling: both हज़ार and हजार appear as references within GramVaani, and both ज़्यादा and ज्यादा within MUCS), but this should not affect the numbers by much.

Limitations

• Mode selection: peak accuracy comes from picking the mode per clip (Hindi, mixed-code, or English). When the language mix is unknown, the Hindi mixed-code mode is a safe default: on pure-Hindi audio it produces pure-Hindi transcripts with no measured accuracy loss, and on mixed audio it handles the code-switching. Automatic language detection is also supported: let the model generate the language token and inject mixed-code after it (see Usage).
• Clip length: evaluated on clips ≤ 30 s; longer audio should be chunked (standard Whisper practice).
• Hindi–English only; other Indic languages are out of scope for this release.

Intended use

Transcription of Hindi and Hindi–English code-switched speech: voice assistants, contact-center analytics, media captioning, and speech data pipelines.

Model details

• Architecture: Whisper large-v3 (encoder–decoder, 1.55B params) + mixed-code mode
• Languages: Hindi (hi), English (en), Hindi–English code-switch
• Sample rate: 16 kHz input
• I/O: ≤30 s audio per window → text
• License: Apache 2.0

Attribution

We thank Gram Vaani for permission to use the Gram Vaani ASR Challenge 2022 Corpus in training Tara. Gram Vaani builds community-anchored voice based engagement platforms ('Mobile Vaani' clubs) that give underserved and marginalised communities a channel to access information and express themselves.

License

This model is released under the Apache License 2.0.

Citation

If you use Tara in your work, please cite:

@misc{trelis2026tara,
  title  = {Tara: Frontier Hindi Transcription Model},
  author = {{Trelis Research}},
  year   = {2026},
  url    = {https://huggingface.co/Trelis/tara}
}