Vyom TTS MR
Vyom TTS MR is a specialized Text-to-Speech (TTS) model designed for the Marathi language, featuring an expressive male voice. Built by fine-tuning the 3B speech-LLM base model (unsloth/orpheus-3b-0.1-ft) in 16-bit precision, it delivers high-fidelity 24 kHz regional speech synthesis via the SNAC neural vocoder.
Model Details
Model Description
- Developed by: OpenXfi by ATX Labs
- Model Type: Fine-tuned Speech-LLM (Base:
unsloth/orpheus-3b-0.1-ft) - Fine-Tuning Method: 16-Bit LoRA (
r=64,lora_alpha=128, rsLoRA) - Language(s): Marathi (
mr) - Voice Characteristics: Male voice
- Audio Output: 24 kHz WAV (SNAC 24kHz Vocoder)
- License: MIT
Uses
Direct Use
- Generating localized Marathi voiceovers for educational content, audiobooks, and media.
- Building conversational AI agents and regional virtual assistants in Marathi.
- Accessibility applications and reading assistants for Marathi text.
Out-of-Scope Use
- Voice Cloning: The model is not designed or authorized for unauthorized voice cloning or deepfake generation.
- Deceptive Content: Generating misleading audio, unverified synthetic news, or unauthorized impersonations.
Evaluation & Benchmarks
Evaluation Dataset
The model was evaluated using a benchmark dataset of 210 Marathi evaluation prompts divided across three core domains:
- Marathi Complex (60 prompts): Long-form literary text, rare vocabulary, and complex sentence structures.
- Marathi Normal (100 prompts): General everyday conversational Marathi text.
- Marathi Numbers (50 prompts): Sentences containing numerical values presented as digits and spelled-out words.
Evaluation Methodology
All synthesized 24 kHz Marathi audio samples were transcribed using AI4Bharat IndicConformer Multilingual ASR (ai4bharat/indic-conformer-600m-multilingual) to compute Word Error Rate (WER) and Character Error Rate (CER).
Empirical Results
| Category | Prompt Count | Word Error Rate (WER) | Character Error Rate (CER) |
|---|---|---|---|
| Marathi Complex | 60 | 0.1382 | 0.0342 |
| Marathi Normal | 100 | 0.1998 | 0.0554 |
| Marathi Numbers | 50 | 0.3170 | 0.1965 |
| Overall Average | 210 | 0.2101 | 0.0829 |
Bias, Risks, and Limitations
- Numerical Formatting: Basic numerical functionality provided. Optimal synthesis performance is achieved when numbers are written out as text words in Marathi (e.g.,
"एक हजार पाचशे") rather than raw numeric digits (e.g.,"१५००"). - Out-of-Vocabulary Terms: Unusual proper nouns, complex Sanskritized loanwords, or rare foreign technical terms may introduce minor acoustic artifacts.
How to Get Started with the Model
Prerequisites
Install the required dependencies:
pip install torch transformers snac soundfile
Path A: Direct Model Generation (generate_speech)
import os, torch, soundfile as sf
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "atx-labs/voice-marathi-male"
hf_token = os.environ.get("HF_TOKEN")
tokenizer = AutoTokenizer.from_pretrained(repo_id, token=hf_token)
model = AutoModelForCausalLM.from_pretrained(
repo_id, trust_remote_code=True, torch_dtype=torch.bfloat16, token=hf_token
).to("cuda").eval()
# Synthesize speech directly
waveform = model.generate_speech("नमस्कार, तुमचे स्वागत आहे.", tokenizer)
# Save 24 kHz WAV audio
sf.write("output_mr.wav", waveform, 24000)
print("✅ Audio generated and saved to output_mr.wav!")
Path B: transformers Pipeline
import os, torch, soundfile as sf
from transformers import pipeline
repo_id = "atx-labs/voice-marathi-male"
hf_token = os.environ.get("HF_TOKEN")
# Initialize TTS pipeline
tts = pipeline("text-to-speech", model=repo_id, trust_remote_code=True, device=0, token=hf_token)
# Synthesize speech
output = tts("नमस्कार, तुमचे स्वागत आहे.")
# Save 24 kHz WAV audio
sf.write("output_mr.wav", output["audio"], output["sampling_rate"])
print(" Audio generated and saved to output_mr.wav!")
Contributors
- Mrugaja Joshi — @MrugajaJ
- Guruprasad Pathak — @grpathak22
Acknowledgements
Special thanks to Modal for providing the compute infrastructure to fine-tune, evaluate, and deploy our models.
About OpenXFI
Vyom TTS is released as part of OpenXFI, an open-source Indic AI initiative by ATX Labs. OpenXFI builds and openly releases foundational models, benchmarks, and datasets for Indian languages — with the goal of making high-quality Indic AI accessible to researchers, developers, and institutions across the country.
Explore the wider release, including our Indic evaluation benchmark and open datasets, at openxfi.com.
Citation
If you use this model in your research or applications, please cite both the model and the OpenXFI initiative:
@misc{vyomttsxfi2026,
title = {Vyom TTS: Open Indic Text-to-Speech Models},
author = {Joshi, Mrugaja and Pathak, Guruprasad},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/atx-labs}},
note = {Released as part of OpenXFI by ATX Labs. Compute provided by Modal (https://modal.com).}
}
@misc{openxfi2026,
title = {OpenXFI: Shaping India's AI, Together},
author = {{ATX Labs}},
year = {2026},
url = {https://openxfi.com}
}
We also encourage citing the upstream work this model builds on — the Orpheus base model and the SNAC neural audio codec.
Contact
- Website: openxfi.com
- ATX Labs: atxlabs.ai
- Issues & feedback: open a discussion on this model's Community tab