quipus-0.6-speechv1

Model Details

Model Description

This is an open-source Text-to-Speech model developed by HyperneuronAI. The model is designed to generate natural speech from text and currently supports Hindi, Assamese, Punjabi, and Kannada. Quipus is 2x Faster than realtime and lower latency TTFB ~120ms.

⚡Quipus Performance

Sneak Peak

In model Inference performance

🗺️ Roadmap

• 🌏 17+ Indian Languages
• 😊 Emotion-aware Speech
• 🌍 Arabic & English support as part of worldwide contribution
• ⚡ Streaming Backend Optimizations
• Training and Inference Code

The model uses a Qwen3 backbone and is intended for research, experimentation, and building voice AI applications. Users are free to fine-tune the model for custom voices and additional languages.

Voice cloning capabilities are not provided with this release to encourage responsible AI usage.

• Developed by: HyperneuronAI
• Funded by: HyperneuronAI
• Shared by: HyperneuronAI
• Model type: Text-to-Speech
• Backbone: Qwen3
• Languages: Hindi, Assamese, Punjabi, Kannada
• License: MIT

Audio Samples

🎙️ Available Voices

Model Sources

• Repository: Realtime optimized streaming inference code is planned for a future release.
• Demo: Coming soon.

Uses

Direct Use

This model can be used for Text-to-Speech generation in voice AI applications, including:

Fine-Tuning

Users may fine-tune this model for:

• New speakers
• Domain-specific speech styles
• Additional Indian languages
• Custom application-specific voices

Out-of-Scope Use

This model should not be used for unethical, harmful, deceptive, or illegal purposes, including but not limited to:

• Impersonation without consent
• Fraudulent voice generation
• Misinformation or manipulation
• Harassment or abuse
• Any use that violates applicable laws or platform policies

HyperneuronAI is not responsible for misuse of this model by third parties.

How to Get Started

vLLM code examples.

#Install libraries if not installed
# Download vlllm based on cuda version
#!pip install vllm==0.19.0
#!pip install -q soundfile

# ==========================================
# vLLM SERVING - Colab-safe audio saving
# ==========================================
"""
You can use vllm for inferencing as it provides optimised control over cuda profiling and inturn better response time.
"""
import time
import re
import torch
import numpy as np
import soundfile as sf
from IPython.display import Audio, display

from vllm import LLM, SamplingParams
from snac import SNAC

VLLM_MODEL_PATH = "hyperneuronAILabs/quipus-0.6-speechv1"
DEFAULT_SPEAKER = "Anvita"
FRAME_LAYER_PATTERN = [0, 1, 2, 2, 1, 2, 2]
SAMPLE_RATE = 24000

llm = LLM(
    model=VLLM_MODEL_PATH,
    dtype="bfloat16",
    max_model_len=2048,
    gpu_memory_utilization=0.20,
    enforce_eager=False,
)

tok = llm.get_tokenizer()

AUDIO_END_ID = tok.convert_tokens_to_ids("<audio_end>")

snac_decoder = (
    SNAC.from_pretrained("hubertsiuzdak/snac_24khz")
    .eval()
    .cuda()
)

def build_prompt_prefix(speaker, text):
    return f"{speaker}: {text} <audio_start> "

def quipus_tts(
    text,
    speaker=DEFAULT_SPEAKER,
    out="vllm_out.wav",
    temperature=0.7,
    top_p=0.9,
):
    prompt = build_prompt_prefix(speaker, text)

    sampling_params = SamplingParams(
        temperature=temperature,
        top_p=top_p,
        repetition_penalty=1.1,
        max_tokens=1024,
        stop_token_ids=[AUDIO_END_ID],
    )

    t0 = time.time()
    result = llm.generate([prompt], sampling_params)
    gen_time = time.time() - t0

    out_ids = list(result[0].outputs[0].token_ids)
    toks = tok.convert_ids_to_tokens(out_ids)

    parsed = []
    for token in toks:
        match = re.fullmatch(r"<snac_l(\d+)_c(\d+)>", token)
        if match:
            parsed.append((int(match.group(1)), int(match.group(2))))

    l0, l1, l2 = [], [], []
    i = 0

    while i + 7 <= len(parsed):
        window = parsed[i:i + 7]

        if [layer for layer, _ in window] == FRAME_LAYER_PATTERN:
            codes = [code for _, code in window]

            l0.append(codes[0])
            l1.extend([codes[1], codes[4]])
            l2.extend([codes[2], codes[3], codes[5], codes[6]])

            i += 7
        else:
            i += 1

    if not l0:
        print("No valid SNAC frames from vLLM output.")
        return None

    with torch.inference_mode():
        wav = snac_decoder.decode(
            [
                torch.tensor([l0], dtype=torch.long, device="cuda"),
                torch.tensor([l1], dtype=torch.long, device="cuda"),
                torch.tensor([l2], dtype=torch.long, device="cuda"),
            ]
        )

    audio_sec = wav.shape[-1] / SAMPLE_RATE

    pcm = wav.detach().squeeze().float().cpu().numpy()
    pcm = np.clip(pcm, -1.0, 1.0)

    sf.write(out, pcm, SAMPLE_RATE)

    n = len(out_ids)

    display(Audio(out, rate=SAMPLE_RATE))

    return out

quipus_tts(
    "यह मॉडल स्पीच-टू-टेक्स्ट मॉडल है , जिसे निखिल ने विकसित किया है। এই মডেলটো হৈছে স্পিচ টু টেক্সট মডেল, নিখিলে বিকশিত কৰিছে",
    speaker=DEFAULT_SPEAKER,
)

Training Details

Training Data

The model was trained and fine-tuned on multilingual speech data covering Hindi, Assamese, Punjabi, and Kannada.

More details about the dataset composition, duration, speakers, and preprocessing pipeline will be added in a future update.

Training Procedure

The model was fine-tuned for text-to-speech generation using a Qwen3-based architecture.

More details about training configuration, tokenizer setup, audio codec/token representation, and optimization strategy will be added later.

Training Hyperparameters

Training hyperparameters will be added in a future update.

Evaluation

Testing Data

The model has been tested on multilingual text prompts across Hindi, Assamese, Punjabi, and Kannada.

A detailed benchmark set will be released in a future update.

Metrics

Formal evaluation metrics such as MOS, speaker similarity, intelligibility, word error rate, and latency benchmarks are not yet published.

Results

Evaluation results will be added after broader testing and benchmarking.

Technical Specifications

Model Architecture and Objective

This is a Text-to-Speech model with a Qwen3 backbone. The model is optimized to generate speech from input text in supported Indian languages.

Further architecture details will be added in future documentation.

Compute Infrastructure

Compute details will be added in a future update.

Hardware

Hardware details will be added in a future update.

Software

Software and inference dependencies will be added with the official inference code.

Limitations

• The model currently supports Hindi, Assamese, Punjabi, and Kannada.
• Output quality may vary depending on language, text normalization, punctuation, and input style.
• The model may struggle with code-mixed text, rare words, abbreviations, numerals, and domain-specific terminology.
• Voice cloning is not included in this release.
• Realtime streaming inference code is planned but not included yet.

Ethical Considerations

This model is released to support open-source development of Indian language voice AI. Users should ensure responsible deployment, obtain consent where required, and avoid deceptive or harmful applications.

Citation

Citation details will be added in a future release.

Authors

🤝 Contact

For questions, collaborations, or contributions, please contact HyperneuronAI

📧 support@hyperneuron.in

🌐 https://www.hyperneuronai.com

🤗 https://huggingface.co/hyperneuronAILabs