Orpheus-3B Turkish TTS (v2)

Fine-tuned unsloth/orpheus-3b-0.1-pretrained for Turkish text-to-speech using LoRA on a SNAC 24 kHz audio-token vocabulary. Trained on Red Hat OpenShift AI with Kubeflow Trainer v2 (TrainJob) · RHOAIENG-62326.

Use case: airline-style Turkish announcements where general-purpose TTS lacks Turkish phonology.

Model hierarchy & specs

unsloth/orpheus-3b-0.1-pretrained     # English-biased Orpheus base (frozen)
        └── LoRA adapters (r=32, α=64)  # ~185M trainable params
                └── merged → this repo  # full weights, Turkish TTS

Orpheus frames TTS as causal LM over SNAC audio tokens: Turkish text → control tokens → autoregressive codec tokens → 24 kHz waveform.

Dataset specs

Training stack

How this model was trained

End-to-end pipeline (reproducible from GitHub manifests):

Hyperparameters (v2 run)

Training approach

TTS is next-token prediction on interleaved text + SNAC audio tokens. Only audio tokens contribute to loss.

1. Prompt format — Orpheus control tokens wrap Turkish text:

ids = tokenizer.encode(text, add_special_tokens=False) + [TOK_EOT]
prompt = [TOK_SOH] + ids + [TOK_EOH, TOK_SOA, TOK_SOS]
# model autoregressively predicts SNAC tokens until TOK_EOA

2. Audio-only loss masking — prompt positions excluded from cross-entropy:

sos_idx = input_ids.index(TOK_SOS)
labels = [-100] * (sos_idx + 1) + input_ids[sos_idx + 1:]

3. LoRA on frozen backbone:

lora_cfg = LoraConfig(
    task_type=TaskType.CAUSAL_LM, r=32, lora_alpha=64, lora_dropout=0.05,
    target_modules=["q_proj","k_proj","v_proj","o_proj",
                    "gate_proj","up_proj","down_proj"],
)
model = get_peft_model(model, lora_cfg)

4. Distributed TrainJob — env-driven config from trainjob-orpheus.yaml:

# manifests/trainjob-orpheus.yaml (excerpt)
runtimeRef: torch-distributed-cuda130-torch210-py312
numNodes: 2
numProcPerNode: gpu
LEARNING_RATE: "2e-5"
NUM_EPOCHS: "8"
MAX_TRAIN_SAMPLES: "20000"
BATCH_SIZE: "2"
GRAD_ACCUM: "4"
LORA_R: "32"
LORA_ALPHA: "64"

MLflow tracking

Results & MLflow

Post-train benchmark (eval-v4): pretrained base vs merged final/ · Whisper-small · temp=0.3, top_p=0.9, rep_penalty=1.15 · finetuned uses length-scaled min_new_tokens; baseline capped at max_new_tokens=400 (English pretrained rarely emits end-of-audio on Turkish).

MLflow: orpheus-turkish-tts · train 6804b44335f347849f26da2736aa73df · eval-v4 50c177d696164b4a83169292b2109b4e

Baseline comparison: Audio labeled B is unsloth/orpheus-3b-0.1-pretrained (English-pretrained) given the same Turkish text and generation settings. F is this checkpoint. Baseline WER/CER vs Turkish references are included for completeness — listen to the pairs below for perceptual comparison.

MLflow UI (training run)

Traces — per-inference span breakdown (tokenise → model_generate → snac_decode) with RTF and duration:

Artifacts — step-indexed WAV + spectrograms browsable in the MLflow UI (example: flight_announce @ step 8800):

Exported metric charts

Per-sentence benchmark (eval/eval_results.json) — eval-v4:

Audio samples

9 curated phrases (ΔCER ≥ 5pp, finetuned CER ≤ 0.85, duration ≥ 1.5s). Excluded: safety only. B = baseline (unsloth/orpheus-3b-0.1-pretrained, English-only) · F = finetuned. Baseline clips use capped generation (max_new_tokens=400) because the pretrained model rarely emits end-of-audio on OOD Turkish and would otherwise ramble for 10–15s of unrelated speech.

Welcome — istanbul'a hoş geldiniz. · F-CER 0.13

B

F

Flight — sayın yolcularımız, uçuşumuz yaklaşık iki saat sürecektir. · F-CER 0.19

B

F

Directions — düz gidin, sonra sağa dönün ve köprüyü geçin. · F-CER 0.22

B

F

News intro — ana haberlere geçmeden önce önemli bir duyurumuz var. · F-CER 0.15

B

F

Farewell — teşekkür ederiz, iyi yolculuklar dileriz. · F-CER 0.29

B

F

Weather — bugün istanbul'da hava bulutlu ve serin olacak. · F-CER 0.28

B

F

Question — yarın toplantıya katılabilir misiniz? · F-CER 0.51

B

F

Tech — yapay zeka teknolojisi her geçen gün gelişmeye devam ediyor. · F-CER 0.43

B

F

Emergency — acil durum! lütfen binayı derhal tahliye edin. · F-CER 0.61

B

F

Quick Start

pip install torch transformers peft soundfile librosa snac

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from snac import SNAC

MODEL = "AbDhumal/orpheus-3b-turkish-tts-v2"
V = 128_256
TOK_SOH, TOK_EOH, TOK_SOA, TOK_SOS, TOK_EOA, TOK_EOT = V+3, V+4, V+5, V+1, V+6, V+9

tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, torch_dtype=torch.bfloat16).eval().cuda()
snac = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").eval().cuda()

text = "istanbul'a hoş geldiniz."
ids = tokenizer.encode(text, add_special_tokens=False) + [TOK_EOT]
prompt = [TOK_SOH] + ids + [TOK_EOH, TOK_SOA, TOK_SOS]
min_new = max(80, len(text) * 7)  # match evaluate_orpheus.py
out = model.generate(torch.tensor([prompt]).cuda(), max_new_tokens=1500, min_new_tokens=min_new,
    do_sample=True, temperature=0.3, top_p=0.9, repetition_penalty=1.15, eos_token_id=TOK_EOA)
# SNAC decode → 24 kHz WAV (see scripts/evaluate_orpheus.py)

Reproduction

git clone https://github.com/abhijeet-dhumal/oai-tts-finetuning.git
cd oai-tts-finetuning/orpheus-tts

# ConfigMap (scripts) + PVC
oc kustomize . | oc apply -f - -n <namespace>

# Train (v2 hyperparameters)
oc apply -f manifests/trainjob-orpheus.yaml

# After training — merge best LoRA checkpoint
python scripts/save_final_orpheus.py   # or run on cluster PVC

# Post-train eval
oc apply -f manifests/trainjob-orpheus-eval.yaml

Limitations: PoC · English base cannot speak Turkish · Whisper metrics noisy · Not production-certified.

License: unsloth/orpheus-3b-0.1-pretrained