VibeVoice-Realtime-0.5B — GGUF

GGUF conversion of microsoft/VibeVoice-Realtime-0.5B for use with CrispASR.

Model variants

The Q8_0/Q4_K files keep the diffusion prediction head, connectors and EOS classifier at full precision (only the two Qwen2 backbones are quantized). The head runs under classifier-free guidance over 20 solver steps, so quantizing it could push the first frames onto a wrong trajectory that decodes as a brief non-speech "music"/hum onset before the voice; keeping it at full precision avoids that at a small size cost.

Voice prompts

A voice prompt is required for TTS. Each .gguf voice pack ships pre-computed KV caches that establish a fixed speaker identity (the realtime variant is not WAV-cloning — for runtime cloning, use cstr/vibevoice-1.5b-GGUF instead).

This repo bundles all 25 demo voices from microsoft/VibeVoice@main demo/voices/streaming_model/ — same MIT license as the model.

Each voice pack is ~2-6 MB. The vibevoice-voice-emma.gguf filename is kept as the legacy default (referenced by crispasr -m auto --backend vibevoice-tts in CrispASR's auto-download manifest); vibevoice-voice-en-Emma_woman.gguf is the canonical upstream-named copy.

Usage

# English with Emma
crispasr --backend vibevoice-tts \
    -m vibevoice-realtime-0.5b-q4_k.gguf \
    --voice vibevoice-voice-emma.gguf \
    --tts "Hello, how are you today?" \
    --tts-output hello.wav

# Japanese with jp-Spk1_woman
crispasr --backend vibevoice-tts \
    -m vibevoice-realtime-0.5b-q4_k.gguf \
    --voice vibevoice-voice-jp-Spk1_woman.gguf \
    --tts "こんにちは、これは日本語の音声テストです。" \
    --tts-output jp.wav

Output: 24 kHz mono WAV. Use crispasr -m auto --backend vibevoice-tts to auto-download the model + the default Emma voice.

Runtime note: CrispASR keeps the initial sigma-VAE decoder samples for the realtime model. Builds after 2026-06-18 also avoid spread-spectrum watermark boundary amplification, fixing the earlier start-click artifact seen in some short generated WAVs.

Architecture

VibeVoice-Realtime-0.5B is a streaming text-to-speech model:

• Base LM: 4-layer Qwen2 (text encoding with voice context)
• TTS LM: 20-layer Qwen2 (speech conditioning, autoregressive)
• Prediction head: 4 AdaLN + SwiGLU layers (flow matching denoiser)
• DPM-Solver++: 20-step 2nd-order midpoint solver (cosine schedule, v-prediction)
• Classifier-Free Guidance: dual KV cache, cfg_scale=3.0
• sigma-VAE decoder: 7-stage transposed ConvNeXt (3200× upsample to 24 kHz)
• EOS classifier: automatic length detection

Quality verification

All quantisations produce exact ASR round-trip matches on English:

Decoder-start parity was also checked against the official PyTorch decoder by replaying saved realtime latents: the C++ raw decoder output starts smoothly and matches the reference onset profile before CLI watermarking and WAV serialization.

Conversion

Model:

python models/convert-vibevoice-to-gguf.py \
    --input microsoft/VibeVoice-Realtime-0.5B \
    --output vibevoice-realtime-0.5b-tts-f16.gguf \
    --include-decoder

build/bin/crispasr-quantize vibevoice-realtime-0.5b-tts-f16.gguf \
    vibevoice-realtime-0.5b-q4_k.gguf q4_k

Voice packs (one per upstream .pt):

python models/convert-vibevoice-voice-to-gguf.py \
    --input demo/voices/streaming_model/en-Carter_man.pt \
    --output vibevoice-voice-en-Carter_man.gguf

Attribution

• Original model: microsoft/VibeVoice-Realtime-0.5B (MIT) — Microsoft Research.
• Voice packs: demo voices from microsoft/VibeVoice@main (MIT).
• GGUF + ggml runtime: CrispStrobe/CrispASR — see src/vibevoice.cpp, models/convert-vibevoice-to-gguf.py, and models/convert-vibevoice-voice-to-gguf.py.

License

MIT (same as the upstream model and voice prompts).

Provenance and EU AI Act Art. 53 note

• Upstream model: microsoft/VibeVoice-Realtime-0.5B — published by microsoft.
• Upstream licence: mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.