indic-transliterate (prototype)
Roman/Hinglish/code-mixed → native Indic script transliteration, built to sit in front of a TTS pipeline (pronunciation-preserving transliteration, not translation — e.g. "EMI" → "ईएमआई", not "किस्त").
⚠️ Status: early prototype, not production-ready
This checkpoint was trained on a deliberately small, capped slice of data (3,000 word pairs per language from AI4Bharat Aksharantar, ~13k examples total after language-temperature sampling and customer-support-data mixing) for 3 epochs, as an end-to-end pipeline validation step — not a production training run.
Measured on a held-out test set:
| Metric | Value |
|---|---|
| Character Error Rate | 91% |
| Exact match | 0% |
| Script validity | 99.9% |
Script validity (99.9%) shows the model reliably learned which script to emit per language tag — the architecture and data pipeline are wired correctly. Character accuracy (91% CER) is not usable yet — that needs a full-scale training run (full per-language data volume, more epochs) before this should be used for anything beyond pipeline testing.
What this is (and isn't)
- Is: a from-scratch, ~7.7M-parameter character-level Transformer
encoder-decoder (
torch.nn.Transformer-based), with<2xx>language-tag tokens (same convention as IndicXlit/NLLB/mBART), trained via Hugging Facetransformers. - Isn't: AI4Bharat's IndicXlit checkpoint or its fairseq architecture.
IndicXlit's fairseq codebase is not compatible with Python 3.12 (mutable
dataclass-default fields fairseq declares are hard-rejected by Python
3.11+'s
dataclassesmodule, across multiple files, not a single patchable spot) — see the training repository's README for the full writeup. This model uses the same task framing and tagging convention on a maintained stack instead of vendoring IndicXlit's literal weights.
Training data
AI4Bharat Aksharantar
— word-level Roman↔native pairs, 21 languages. Aksharantar's license is
mixed (CC-BY for manually-collected rows, CC0 for mined rows); this model
card carries cc-by-4.0 as the safe upper bound. See the training repo's
README for the full licensing note, including why Google Dakshina (CC BY-SA,
share-alike) was not used for this checkpoint.
Usage
from transliteration.inference import TransliterationEngine
engine = TransliterationEngine.from_checkpoint("path/to/this/checkpoint")
engine.transliterate("mera emi pending hai", language="hi")
Full pipeline code (data download/validation/training/evaluation/inference/ TTS integration): see the accompanying repository files.
Intended use
Pipeline validation and further fine-tuning. Not intended for production transliteration until retrained at full data scale — see Known Limitations in the training repo's README.