🎙️ Arabic_TTS by theBOrganization

🌟 Overview

Arabic_TTS is a state-of-the-art Text-to-Speech model specifically optimized for the Arabic language. Built on the powerful VibeVoice architecture, this model goes beyond standard TTS by offering zero-shot voice cloning.

Simply provide a short reference audio clip, and the model will synthesize your input text in Arabic while perfectly capturing the timbre, tone, and unique characteristics of the reference speaker.

✨ Key Features

• 🗣️ Native Arabic Fluency: Handles complex Arabic morphology, diacritics (Tashkeel) with high fidelity.
• 🎭 Zero-Shot Voice Cloning: Clone any voice from a single reference audio sample without fine-tuning.
• 🎛️ Controllable Generation: Built-in Classifier-Free Guidance (CFG) and temperature sampling for expressive and diverse outputs.
• ⚡ Optimized Inference: Supports CUDA (bfloat16), Apple Silicon MPS (float32), and CPU for maximum efficiency.

🎧 Audio Samples

Listen to the quality of the model below. We compare the original reference voice against the synthetic generated output.

🛠️ Usage & Inference

Prerequisites

Ensure you have PyTorch installed. You will also need the vibevoice library.

pip install torch torchaudio
# Install vibevoice (adjust based on your specific package distribution)
pip install vibevoice 

Inference Code

Below is the complete script to load the model, process the text and reference audio, and generate high-quality 24kHz speech.

import os
import torch
import torchaudio as ta
from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor
from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference

# --- Configuration ---
model_path = "theBOrganization/Arabic_TTS"
text = "SPEAKER 1: مرحبا، هذا مثال على توليد الصوت باللغة العربية باستخدام نموذج جديد."
output_path = "output.wav"

# Device selection with fallback
device = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
print(f"🚀 Loading model on {device}...")

# Load Processor
processor = VibeVoiceProcessor.from_pretrained(model_path)

# Set dtype and attention implementation based on device
if device == "mps":
    dtype = torch.float32
elif device == "cuda":
    dtype = torch.bfloat16
else:
    dtype = torch.float32  # Note: float32 is recommended for CPU stability
    
attn_impl = "sdpa"

# Load Model
model = VibeVoiceForConditionalGenerationInference.from_pretrained(
    model_path,
    torch_dtype=dtype,
    attn_implementation=attn_impl,
    device_map=device if device != "mps" else None,
)

if device == "mps":
    model.to("mps")
    
model.eval()
model.set_ddpm_inference_steps(10) # 10 steps for DDPM inference

# --- Prepare Inputs ---
reference_voice = "prompt.wav" # Replace with your reference audio path
print(f"🎙️ Using reference voice: {reference_voice}")

inputs = processor(
    text=[text],
    voice_samples=[[reference_voice]],  # Batch of one speaker list
    return_tensors="pt",
    padding=True,
)

# Move tensors to the correct device
inputs = {
    k: v.to(device) if torch.is_tensor(v) else v
    for k, v in inputs.items()
}

# --- Generate Audio ---
print("🎧 Generating audio...")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        cfg_scale=1.3,
        tokenizer=processor.tokenizer,
        generation_config={
            'do_sample': True, 
            'temperature': 0.5
        },
    )

# --- Save Output ---
audio = outputs.speech_outputs[0].cpu().float()  # shape: [1, T] or [T]
if audio.ndim == 1:
    audio = audio.unsqueeze(0).float()
    
sample_rate = 24000
ta.save(output_path, audio, sample_rate)
print(f"✅ Successfully saved audio to {output_path}")

📝 Input Formatting

The model expects text to be prefixed with a speaker identifier.

• Format: SPEAKER X: <Your Arabic Text Here>
• Example: SPEAKER 1: أهلاً وسهلاً بكم في موقعنا.

⚙️ Model Details

⚠️ Limitations & Ethical Considerations

While Arabic_TTS produces highly realistic speech, users must adhere to responsible AI practices:

1. Consent: Do not use the zero-shot voice cloning feature to clone voices without the explicit consent of the speaker.
2. Misinformation: Do not use this model to generate deceptive audio (deepfakes) for malicious purposes, fraud, or political manipulation.
3. Dialects: While the model excels at Modern Standard Arabic (MSA), extreme regional dialects or heavy code-switching (mixing Arabic and English in the same sentence) may result in slight pronunciation artifacts.

Developed with ❤️ by [theBOrganization]
For inquiries, collaborations, or bug reports, please open an issue on the repository or contact us directly.