Wuro — Text-to-Speech for Bambara and Bomu

Wuro-Nucobiyo-Tanle means Speak Black People(or African) Languages in Bomu is a text-to-speech (TTS) model designed to generate speech in 19 languages.

Description

This model was developed for speech synthesis in 17 African languages.
It uses an autoregressive approach that combines tokenized text with audio codecs.

Supported Languages

• Bambara (bm)
• Bomu (bmq)
• Jula (dyu)
• Dogon (dts)
• Minianka(Mamara) (myk)
• Fulfulde (ffm)
• Konabere (bbo)
• Songhay-Timbuktu (thq)
• Moore (mos)
• Khassonke (kao)
• Senoufo-Shempire (spp)
• Senoufo-Sicite (sep)
• Senoufo-Nyarafolo (sev)
• Senoufo-Djimini (dyi)
• Malinke (mwk)
• Wolof (wol)
• Tamasheq (taq)
• English (en)
• French (fr)

Training Model

This model is a extended version of Panga-Azazia/Wuro-Mali-Tanle.

Training Data

The model was finetuned on dataset built from Bible.

Features

• Text-to-speech generation
• Multilingual support (18 languages)
• Audio generation based on SNAC tokens.

Usage

Installation

pip install unsloth snac
pip install git+https://github.com/sudoping01/bambara-text-normalization.git

Using unsloth

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Panga-Azazia/Wuro-Nucobiyo-Tanle",
    max_seq_length=4096,
    dtype=None,
    load_in_4bit=False,
)

FastLanguageModel.for_inference(model)

Using transformers

tokenizer = AutoTokenizer.from_pretrained("Panga-Azazia/Wuro-Nucobiyo-Tanle", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "Panga-Azazia/Wuro-Nucobiyo-Tanle",
    trust_remote_code=True
)

Normalisation du Bambara

The bambara_normalizer module is used only for Bambara. It is used to normalize certain parts of the text before generation.

For more information on how to use it, see: bambara-text-normalizer

import torch
from snac import SNAC
import soundfile as sf
import numpy as np
from unsloth import FastLanguageModel
from bambara_normalizer import normalize_dates_in_text, normalize_measurements_in_text, normalize_numbers_in_text, normalize_times_in_text

CODE_START_TOKEN_ID = 128257
CODE_END_TOKEN_ID = 128258
CODE_TOKEN_OFFSET = 128266
SNAC_MIN_ID = 128266
SNAC_MAX_ID = 156937
SNAC_TOKENS_PER_FRAME = 7

SOH_ID = 128259
EOH_ID = 128260
SOA_ID = 128261
BOS_ID = 128000
TEXT_EOT_ID = 128009

tokeniser_length = 128256
pad_token = tokeniser_length + 7

# -------------------------------------------------------------------------------------------------------------
# Normalizer from sudoping01 : see https://github.com/sudoping01/bambara-text-normalization.git for more info;
# --------------------------------------------------------------------------------------------------------------
def normalize_text(text: str) -> str:
    """Normalize text."""
    text = normalize_dates_in_text(text)
    text = normalize_times_in_text(text)
    text = normalize_measurements_in_text(text)
    text = normalize_numbers_in_text(text)
    return text

def build_prompt(tokenizer, lang: str, text: str) -> str:
   
    soh_token = tokenizer.decode([SOH_ID])
    eoh_token = tokenizer.decode([EOH_ID])
    soa_token = tokenizer.decode([SOA_ID])
    sos_token = tokenizer.decode([CODE_START_TOKEN_ID])
    eot_token = tokenizer.decode([TEXT_EOT_ID])
    bos_token = tokenizer.bos_token

    if lang.lower() == "bambara" :
        text = normalize_text(text)
    

    formatted_text = f"<description=Read the following text in {lang}> {text}"


    prompt = (
        soh_token + bos_token + formatted_text + eot_token +
        eoh_token + soa_token + sos_token
    )
    return prompt



def extract_snac_codes(token_ids: list) -> list:
    try:
        eos_idx = token_ids.index(CODE_END_TOKEN_ID)
    except ValueError:
        eos_idx = len(token_ids)

    return [
        token_id for token_id in token_ids[:eos_idx]
        if SNAC_MIN_ID <= token_id <= SNAC_MAX_ID
    ]


def unpack_snac_from_7(snac_tokens: list) -> list:
    """Unpack 7-token SNAC frames to 3 hierarchical levels."""
    if snac_tokens and snac_tokens[-1] == CODE_END_TOKEN_ID:
        snac_tokens = snac_tokens[:-1]

    frames = len(snac_tokens) // SNAC_TOKENS_PER_FRAME
    snac_tokens = snac_tokens[:frames * SNAC_TOKENS_PER_FRAME]

    if frames == 0:
        return [[], [], []]

    l1, l2, l3 = [], [], []

    for i in range(frames):
        slots = snac_tokens[i * 7:(i + 1) * 7]
        l1.append((slots[0] - CODE_TOKEN_OFFSET) % 4096)
        l2.extend([
            (slots[1] - CODE_TOKEN_OFFSET) % 4096,
            (slots[4] - CODE_TOKEN_OFFSET) % 4096,
        ])
        l3.extend([
            (slots[2] - CODE_TOKEN_OFFSET) % 4096,
            (slots[3] - CODE_TOKEN_OFFSET) % 4096,
            (slots[5] - CODE_TOKEN_OFFSET) % 4096,
            (slots[6] - CODE_TOKEN_OFFSET) % 4096,
        ])

    return [l1, l2, l3]


snac_model = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").eval()

def main(lang = "Bomu", text = "bwe wa wuro." , temp = .4, top_p = .9,
         max_tokens = 2048, min_tokens = 28, rp = 1.2):

  
    if lang not in ["Bambara", "Bomu", "Dogon", "Minianka"] : lang = lang.lower()

    prompt = build_prompt(tokenizer, lang, text)
    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)

    if torch.cuda.is_available():
        inputs = {k: v.to("cuda") for k, v in inputs.items()}

    with torch.inference_mode():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,  
            min_new_tokens=min_tokens, 
            temperature=temp,
            top_p=top_p,
            repetition_penalty=rp,  
            do_sample=True,
            eos_token_id=CODE_END_TOKEN_ID,  
            pad_token_id=tokenizer.pad_token_id,
        )

    generated_ids = outputs[0, inputs['input_ids'].shape[1]:].tolist()

    if CODE_END_TOKEN_ID in generated_ids:
        eos_position = generated_ids.index(CODE_END_TOKEN_ID)
        print(f" EOS token found at position {eos_position}/{len(generated_ids)}")

    snac_tokens = extract_snac_codes(generated_ids)

    print(f"Extracted {len(snac_tokens)} SNAC tokens")


    snac_count = sum(1 for t in generated_ids if SNAC_MIN_ID <= t <= SNAC_MAX_ID)
    other_count = sum(1 for t in generated_ids if t < SNAC_MIN_ID or t > SNAC_MAX_ID)
    print(f"   SNAC tokens in output: {snac_count}")
    print(f"   Other tokens in output: {other_count}")

    # Check for SOS token
    if CODE_START_TOKEN_ID in generated_ids:
        sos_pos = generated_ids.index(CODE_START_TOKEN_ID)
        print(f"   SOS token at position: {sos_pos}")
    else:
        print(f"   No SOS token found in generated output!")

    if len(snac_tokens) < 7:
        print("Error: Not enough SNAC tokens generated")
        return

    # Unpack SNAC tokens to 3 hierarchical levels
    levels = unpack_snac_from_7(snac_tokens)
    frames = len(levels[0])

    print(f"Unpacked to {frames} frames")
    print(f"   L1: {len(levels[0])} codes")
    print(f"   L2: {len(levels[1])} codes")
    print(f"   L3: {len(levels[2])} codes")

    device = "cuda" if torch.cuda.is_available() else "cpu"
    codes_tensor = [
        torch.tensor(level, dtype=torch.long, device=device).unsqueeze(0)
        for level in levels
    ]

    with torch.inference_mode():
        z_q = snac_model.quantizer.from_codes(codes_tensor)
        audio = snac_model.decoder(z_q)[0, 0].cpu().numpy()

    # Trim warmup samples (first 2048 samples)
    if len(audio) > max_tokens:
        audio = audio[max_tokens:]

    duration_sec = len(audio) / 24000
    print(f"Audio generated: {len(audio)} samples ({duration_sec:.2f}s)")


    output_file = f"{lang}-{text}.wav"
    sf.write(output_file, audio, 24000)

Examples

A Bomu audio sample.

main(
    lang = "Bomu",
    text="Ho lana a sinciri cinnu, ho a ziya berobe nɛ zeni be nɛ mi, a nucoro yire ɓɛɛ da mi."
)

A Fulfulde audio sample.

main(
    lang = "fulfulde",
    text = "Goonɗinal yo taƴorde heɓan ko fawi jikke e mum e hoolaade ko yiyaali ana woodi."
  )

A Wolof audio sample.

main(
    lang = "wolof",
    text= "iniwersite bu ndar, ëmb na juróorum-ñaari bërëbu tàggat ak gëstu.",
  )

A Bambara audio sample.

main(
    lang = "Bambara",
    text="Ayiwa, an jigi bɛ fɛn minnu kan, dannaya ye olu jɛyali ye. Dannaya bɛ fɛn yebaliw seereya kɛ."
  )

A Konabere audio sample.

main(
    lang = "konabere",
    text="Laɩnapɩ tɩɛɛ n’ɩ kɔ nʋ: ŋma daŋ ma ma tɩgɛɩmapɩ ti, ŋmɔ na jisigefaa, ŋma fɩrɛɛ ma m’ɩ jaa ko, ŋmɔ na sɩɛrɩpɩ.",
  )

A Khassonke audio sample.

main(
    lang = "khassonke",
    text="Lannaya mu ɲin ne ti xo: Í gigo be fenŋo men xan, wo soto be xoyiyanin í ma; walla í ɲa nte fenŋo men na, í b'a yitanin xo í ɲa be wo la.",
  )

A English audio sample.

main(
    lang = "english",
    text="Now faith is the substance of things hoped for, the evidence of things not seen.",
  )

A Songhay-timbuktu audio sample.

main(
    lang = "songhay-timbuktu",
    text="Naanay ci, ka kow sika hayadiyo ra ka ni go tammaha, nda ka tabatindi hayadiyo ka ni si guna.",
  )

A Moore audio sample.

main(
    lang = "moore",
    text="Wẽnd sẽn pʋlem tõnd vʋʋsgã, tɩ b kãabga ket n yɛsẽ wã, d maan dabeem, tɩ yãmb nin-yend na n wa yet tɩ b basa yẽ kɛɛnga.",
  )

A French audio sample.

main(
    lang = "french",
    text="Or la foi est une ferme assurance des choses qu’on espère, une démonstration de celles qu’on ne voit pas.",
  )

A Fulfulde audio sample.

main(
    lang = "fulfulde",
    text = "Goonɗinal yo taƴorde heɓan ko fawi jikke e mum e hoolaade ko yiyaali ana woodi."
  )

A Dogon audio sample.

main(
    lang = "Dogon",
    text="Kijɛ emɛ dɔmɔ bɛbɛɛdɛin ga diyɛ daanu ye bii gɔ, gagawu gɔ woi. Gagawu, kijɛ iyemɔɔlɛ gɔbe yɔɔin ga diyɛ daanu ye bii.",
  )

A Minianka audio sample.

main(
    lang = "Minianka",
    text="Ayiwa, nʼa daa wʼa wèe wo tadaŋa ki wo yaŋmuyɔ yi keree fiinŋɛ, na wèe pye wèe ya tiin ni li ni na yi wa wèe mu na xɔ. Nʼa daa wʼa wèe pye wèe ya sɔɔ li na na can wa yaŋmuɲabaaya yi wo keree ki ni.",
  )

A jula audio sample.

main(
    lang = "jula",
    text="Lanaya ye min ye, o ye nin ye i be la fɛɛn dɔ la, k'i jigi la a kan k'a sɔrɔ i m'a ye ka ye. Fɔlɔfɔlɔ mɔgɔw ye tɔgɔɲuman sɔrɔ Ala fɛ u ka lanaya kosɔn.",
  )

A Senoufo-Djimini audio sample.

main(
    lang = "Senoufo-Djimini",
    text="Na we kaga jɛn mbe yo we jigi wi yɛn yaraga ŋga na we yaa ki ta, ko ki yɛn tagawa we. Tagawa po pi yɛn yaraga ŋga we woro na yaan, we taga ki na we yo ki yɛn kaselege.",
  )

A Senoufo-Shempire audio sample.

main(
    lang = "Senoufo-Shempire ",
    text="Ɲaha ku ɲyɛ dániyaŋi yɛ? Mà ma sɔ̀nŋɔre taha yaaga na maa ntɛ̀ɛn ná l'e mu sí ku ta, lire ɲyɛ mɛ yaage ku ɲyɛ mu ɲyɛ na ku ɲaa mɛ, maa dá ku na mu à jwo mu ɲyii na ɲyɛ ku na ke, lire li ɲyɛ dániyaŋi.",
  )

A Senoufo-Nyarafolo audio sample.

main(
    lang = "Senoufo-Nyarafolo",
    text="Tɛ́ngɛlɛ ni kɔri wire yē bè tɛ́ngɛ ki nɛ̄ gìi nɛ̄ muɔ nyɛ́nì muɔ sɔ̀ngirɔ taʔa dè, muɔ bé ki taa, bè ki cɛ́n gàa muɔ wè nɛ nyaʔa gèʔ, kiī kányiʔɛ.",
  )

A Malinke audio sample.

main(
    lang = "malinke",
    text="Lannaya dun ye le xo ilan jigi ye hɛn mɛnnu xan, olu jusigilan ne ye, ilan ɲa ye ntɛ hɛn mɛnnu la, olu seeriyali le ye.",
  )

A tamasheq audio sample.

main(
    lang = "tamasheq",
    text="Immun, ălmăɣna-net a-s a-wa dăɣ-ijăr ăwadəm ăṭṭăma, ad-has-umas šund a-s insa data-s; meɣ daɣ, a-wa-dăɣ wăr-ihənnəy s-tiṭṭawen-net, ad-has-umas šund ăsiḍălalăk dăɣ-s."
  )

Other examples

You can also use one language's voice to read text written in another.

A English audio sample. Here the model are reading English using Wolof accent.

main(
    lang = "wolof",
    text="Now faith is the substance of things hoped for, the evidence of things not seen.",
  )