Qwen-3-TTS-12Hz-1.7B-Base-hi-LoRA-Finetuned

LoRA adapters fine-tuned on top of Qwen/Qwen3-TTS-12Hz-1.7B-Base for Hindi using the full AI4Bharat Rasa Hindi split.

Two gender-specific adapters are provided — female and male — in a single repository.

Part of the aguken-ai/qwen3-tts-indic-finetuned collection.

Adapters

Repository Structure

Qwen-3-TTS-12Hz-1.7B-Base-hi-LoRA-Finetuned/
├── README.md
├── config.json                  ┐
├── generation_config.json       │
├── merges.txt                   │
├── model.safetensors            ├─ bundled Qwen3-TTS-1.7B base model
├── preprocessor_config.json     │  (full precision — NOT quantized;
├── tokenizer_config.json        │   loads standalone, no separate download)
├── vocab.json                   │
└── speech_tokenizer/            ┘  12 Hz speech tokenizer (config + weights)
└── adapters/
    ├── hindi_female/
    │   ├── adapter_config.json
    │   ├── adapter_model.safetensors
    │   ├── config.json
    │   ├── ref.wav
    │   ├── ref.txt
    │   └── speaker_embedding.safetensors
    └── hindi_male/
        └── ...

The repo bundles the full, full-precision base model at root (plus speech_tokenizer/), so it loads standalone without separately downloading Qwen/Qwen3-TTS-12Hz-1.7B-Base. No quantization is applied to the base weights.

Each adapter folder contains:

• adapter_config.json — LoRA configuration (rank 16, alpha 32)
• adapter_model.safetensors — LoRA weight delta
• config.json — base model config copy
• ref.wav — reference audio clip for voice cloning (Hindi, that gender)
• ref.txt — transcript of ref.wav (pass as ref_text)
• speaker_embedding.safetensors — learned speaker embedding

Training Details

Quickstart

pip install -U qwen-tts peft transformers torch torchaudio
pip install -U flash-attn --no-build-isolation  # optional, for speed

import torch
import soundfile as sf
from huggingface_hub import snapshot_download, hf_hub_download
from qwen_tts import Qwen3TTSModel
from peft import PeftModel

REPO_ID    = "aguken-ai/Qwen-3-TTS-12Hz-1.7B-Base-hi-LoRA-Finetuned"
COMBO      = "hindi_female"   # or "hindi_male"
LORA_SCALE = 0.35             # tune between 0.1-0.5

BASE_MODEL   = snapshot_download(REPO_ID, allow_patterns=[
    "config.json", "generation_config.json", "merges.txt", "model.safetensors",
    "preprocessor_config.json", "tokenizer_config.json", "vocab.json", "speech_tokenizer/*",
])
ADAPTER_PATH = snapshot_download(REPO_ID, allow_patterns=[f"adapters/{COMBO}/*"]) + f"/adapters/{COMBO}"

model = Qwen3TTSModel.from_pretrained(
    BASE_MODEL, device_map="cuda:0", dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)
model = PeftModel.from_pretrained(model, ADAPTER_PATH)
model.set_adapter_scale(LORA_SCALE)

ref_audio = hf_hub_download(REPO_ID, filename=f"adapters/{COMBO}/ref.wav")
ref_text  = open(hf_hub_download(REPO_ID, filename=f"adapters/{COMBO}/ref.txt"), encoding="utf-8").read().strip()

text = "नमस्ते! मैं Avni बोल रही हूं Synoris Technologies से।"
wavs, sr = model.generate_voice_clone(text=text, language="Hindi", ref_audio=ref_audio, ref_text=ref_text)
sf.write("output.wav", wavs[0], sr)

LoRA Scale Tuning

Speaker IDs

Each adapter was trained with a dedicated speaker_id (rasa_hindi_female, rasa_hindi_male), stored in speaker_embedding.safetensors.

Dataset

Training data is the full Hindi split of AI4Bharat Rasa. Short clips are concatenated into ~60 s samples (audio + transcripts joined), filtered for:

• Clip duration: 1-60 seconds
• Clean text (no URLs, low digit ratio)
• Audio normalized to -20 dB RMS at 24 kHz

License

Apache 2.0 — same as the base model.