Wuro-Mali-Tanle — Text-to-Speech for Malian languages

Wuro-Mali-Tanle means Speak Malian Languages in Bomu is a text-to-speech (TTS) model designed to generate speech in 10 languages.

Description

This model was developed for speech synthesis in 9 Malian languages.
It uses an autoregressive approach that combines tokenized text with audio codecs.

Supported Languages

• Bambara (bm)
• Bomu (bmq)
• Dogon (dts)
• Minianka(Mamara) (myk)
• Fulfulde (ffm)
• Konabere (bbo)
• Songhay-Timbuktu (thq)
• Moore (mos)
• Khassonke (kao)
• English (en)

Training Model

This model is a extended version of RobotsMali/Wuro.

Training Data

The model was finetuned on dataset built from Bible.

Features

• Text-to-speech generation
• Multilingual support (10 languages)
• Audio generation based on SNAC tokens.

Usage

Installation

pip install unsloth snac
pip install git+https://github.com/sudoping01/bambara-text-normalization.git

Using unsloth

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Panga-Azazia/Wuro-Mali-Tanle-v0",
    max_seq_length=4096,
    dtype=None,
    load_in_4bit=False,
)

FastLanguageModel.for_inference(model)

Using transformers

tokenizer = AutoTokenizer.from_pretrained("Panga-Azazia/Wuro-Mali-Tanle-v0", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "Panga-Azazia/Wuro-Mali-Tanle-v0",
    trust_remote_code=True
)

Normalisation du Bambara

The bambara_normalizer module is used only for Bambara. It is used to normalize certain parts of the text before generation.

For more information on how to use it, see: bambara-text-normalizer

import torch
from snac import SNAC
import soundfile as sf
import numpy as np
from bambara_normalizer import normalize_dates_in_text, normalize_measurements_in_text, normalize_numbers_in_text, normalize_times_in_text

CODE_START_TOKEN_ID = 128257
CODE_END_TOKEN_ID = 128258
CODE_TOKEN_OFFSET = 128266
SNAC_MIN_ID = 128266
SNAC_MAX_ID = 156937
SNAC_TOKENS_PER_FRAME = 7

SOH_ID = 128259
EOH_ID = 128260
SOA_ID = 128261
BOS_ID = 128000
TEXT_EOT_ID = 128009

tokeniser_length = 128256
pad_token = tokeniser_length + 7

# -------------------------------------------------------------------------------------------------------------
# Normalizer from sudoping01 : see https://github.com/sudoping01/bambara-text-normalization.git for more info;
# --------------------------------------------------------------------------------------------------------------
def normalize_text(text: str) -> str:
    """Normalize text."""
    text = normalize_dates_in_text(text)
    text = normalize_times_in_text(text)
    text = normalize_measurements_in_text(text)
    text = normalize_numbers_in_text(text)
    return text

def build_prompt(tokenizer, lang: str, text: str) -> str:
   
    soh_token = tokenizer.decode([SOH_ID])
    eoh_token = tokenizer.decode([EOH_ID])
    soa_token = tokenizer.decode([SOA_ID])
    sos_token = tokenizer.decode([CODE_START_TOKEN_ID])
    eot_token = tokenizer.decode([TEXT_EOT_ID])
    bos_token = tokenizer.bos_token

    if lang.lower() == "bambara" :
        text = normalize_text(text)
    

    formatted_text = f"<description=Read the following text in {lang}> {text}"


    prompt = (
        soh_token + bos_token + formatted_text + eot_token +
        eoh_token + soa_token + sos_token
    )
    return prompt



def extract_snac_codes(token_ids: list) -> list:
    try:
        eos_idx = token_ids.index(CODE_END_TOKEN_ID)
    except ValueError:
        eos_idx = len(token_ids)

    return [
        token_id for token_id in token_ids[:eos_idx]
        if SNAC_MIN_ID <= token_id <= SNAC_MAX_ID
    ]


def unpack_snac_from_7(snac_tokens: list) -> list:
    """Unpack 7-token SNAC frames to 3 hierarchical levels."""
    if snac_tokens and snac_tokens[-1] == CODE_END_TOKEN_ID:
        snac_tokens = snac_tokens[:-1]

    frames = len(snac_tokens) // SNAC_TOKENS_PER_FRAME
    snac_tokens = snac_tokens[:frames * SNAC_TOKENS_PER_FRAME]

    if frames == 0:
        return [[], [], []]

    l1, l2, l3 = [], [], []

    for i in range(frames):
        slots = snac_tokens[i * 7:(i + 1) * 7]
        l1.append((slots[0] - CODE_TOKEN_OFFSET) % 4096)
        l2.extend([
            (slots[1] - CODE_TOKEN_OFFSET) % 4096,
            (slots[4] - CODE_TOKEN_OFFSET) % 4096,
        ])
        l3.extend([
            (slots[2] - CODE_TOKEN_OFFSET) % 4096,
            (slots[3] - CODE_TOKEN_OFFSET) % 4096,
            (slots[5] - CODE_TOKEN_OFFSET) % 4096,
            (slots[6] - CODE_TOKEN_OFFSET) % 4096,
        ])

    return [l1, l2, l3]


snac_model = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").eval()

def main(lang = "Bomu", text = "bwe wa wuro." , temp = .4, top_p = .9,
         max_tokens = 2048, min_tokens = 28, rp = 1.2):

  
    if lang not in ["Bambara", "Bomu", "Dogon", "Minianka"] : lang = lang.lower()

    prompt = build_prompt(tokenizer, lang, text)
    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)

    if torch.cuda.is_available():
        inputs = {k: v.to("cuda") for k, v in inputs.items()}

    with torch.inference_mode():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,  
            min_new_tokens=min_tokens, 
            temperature=temp,
            top_p=top_p,
            repetition_penalty=rp,  
            do_sample=True,
            eos_token_id=CODE_END_TOKEN_ID,  
            pad_token_id=tokenizer.pad_token_id,
        )

    generated_ids = outputs[0, inputs['input_ids'].shape[1]:].tolist()

    if CODE_END_TOKEN_ID in generated_ids:
        eos_position = generated_ids.index(CODE_END_TOKEN_ID)
        print(f" EOS token found at position {eos_position}/{len(generated_ids)}")

    snac_tokens = extract_snac_codes(generated_ids)

    print(f"Extracted {len(snac_tokens)} SNAC tokens")


    snac_count = sum(1 for t in generated_ids if SNAC_MIN_ID <= t <= SNAC_MAX_ID)
    other_count = sum(1 for t in generated_ids if t < SNAC_MIN_ID or t > SNAC_MAX_ID)
    print(f"   SNAC tokens in output: {snac_count}")
    print(f"   Other tokens in output: {other_count}")

    # Check for SOS token
    if CODE_START_TOKEN_ID in generated_ids:
        sos_pos = generated_ids.index(CODE_START_TOKEN_ID)
        print(f"   SOS token at position: {sos_pos}")
    else:
        print(f"   No SOS token found in generated output!")

    if len(snac_tokens) < 7:
        print("Error: Not enough SNAC tokens generated")
        return

    # Unpack SNAC tokens to 3 hierarchical levels
    levels = unpack_snac_from_7(snac_tokens)
    frames = len(levels[0])

    print(f"Unpacked to {frames} frames")
    print(f"   L1: {len(levels[0])} codes")
    print(f"   L2: {len(levels[1])} codes")
    print(f"   L3: {len(levels[2])} codes")

    device = "cuda" if torch.cuda.is_available() else "cpu"
    codes_tensor = [
        torch.tensor(level, dtype=torch.long, device=device).unsqueeze(0)
        for level in levels
    ]

    with torch.inference_mode():
        z_q = snac_model.quantizer.from_codes(codes_tensor)
        audio = snac_model.decoder(z_q)[0, 0].cpu().numpy()

    # Trim warmup samples (first 2048 samples)
    if len(audio) > max_tokens:
        audio = audio[max_tokens:]

    duration_sec = len(audio) / 24000
    print(f"Audio generated: {len(audio)} samples ({duration_sec:.2f}s)")


    output_file = f"{lang}-{text}.wav"
    sf.write(output_file, audio, 24000)

Examples

A Bomu audio sample.

main(
    lang = "Bomu",
    text="Ho lana a sinciri cinnu, ho a ziya berobe nɛ zeni be nɛ mi, a nucoro yire ɓɛɛ da mi."
)

A Fulfulde audio sample.

main(
    lang = "fulfulde",
    text = "Goonɗinal yo taƴorde heɓan ko fawi jikke e mum e hoolaade ko yiyaali ana woodi."
  )

A Bambara audio sample.

main(
    lang = "Bambara",
    text="Ayiwa, an jigi bɛ fɛn minnu kan, dannaya ye olu jɛyali ye. Dannaya bɛ fɛn yebaliw seereya kɛ."
  )

A Konabere audio sample.

main(
    lang = "konabere",
    text="Laɩnapɩ tɩɛɛ n’ɩ kɔ nʋ: ŋma daŋ ma ma tɩgɛɩmapɩ ti, ŋmɔ na jisigefaa, ŋma fɩrɛɛ ma m’ɩ jaa ko, ŋmɔ na sɩɛrɩpɩ.",
  )

A Khassonke audio sample.

main(
    lang = "khassonke",
    text="Lannaya mu ɲin ne ti xo: Í gigo be fenŋo men xan, wo soto be xoyiyanin í ma; walla í ɲa nte fenŋo men na, í b'a yitanin xo í ɲa be wo la.",
  )

A English audio sample.

main(
    lang = "english",
    text="Now faith is the substance of things hoped for, the evidence of things not seen.",
  )

A Songhay-timbuktu audio sample.

main(
    lang = "songhay-timbuktu",
    text="Naanay ci, ka kow sika hayadiyo ra ka ni go tammaha, nda ka tabatindi hayadiyo ka ni si guna.",
  )

A Moore audio sample.

main(
    lang = "moore",
    text="Wẽnd sẽn pʋlem tõnd vʋʋsgã, tɩ b kãabga ket n yɛsẽ wã, d maan dabeem, tɩ yãmb nin-yend na n wa yet tɩ b basa yẽ kɛɛnga.",
  )

A Dogon audio sample.

main(
    lang = "Dogon",
    text="Kijɛ emɛ dɔmɔ bɛbɛɛdɛin ga diyɛ daanu ye bii gɔ, gagawu gɔ woi. Gagawu, kijɛ iyemɔɔlɛ gɔbe yɔɔin ga diyɛ daanu ye bii.",
  )

A Minianka audio sample.

main(
    lang = "Minianka",
    text="Ayiwa, nʼa daa wʼa wèe wo tadaŋa ki wo yaŋmuyɔ yi keree fiinŋɛ, na wèe pye wèe ya tiin ni li ni na yi wa wèe mu na xɔ. Nʼa daa wʼa wèe pye wèe ya sɔɔ li na na can wa yaŋmuɲabaaya yi wo keree ki ni.",
  )

Other examples

You can also use one language's voice to read text written in another.

A Bambara audio sample. Here the model are reading Bambara using Bomu accent.

main(
    lang = "Bomu",
    text="Ayiwa, an jigi bɛ fɛn minnu kan, dannaya ye olu jɛyali ye. Dannaya bɛ fɛn yebaliw seereya kɛ."
  )