VibeVoice-ASR-BitNet GGUF

GGUF conversion of microsoft/VibeVoice-ASR-BitNet for use with CrispASR.

Available files

All variants use TQ2_0 (ternary 2-bit) for the LM projection weights. They differ in how aggressively the VAE encoder and LM embedding are quantized. All produce identical transcription output on the JFK benchmark.

Pick any file -- quality is identical across all variants (verified via transcription roundtrip on JFK audio). The aggro variant is 35% smaller than the default with zero degradation.

Model Details

Quantization strategy

The LM projection weights (q/k/v/o/gate/up/down) are BitNet-trained with ternary values {-1, 0, 1}. The converter applies the ternary quantization (s = 1/mean(|w|), round, clamp) and packs into ggml's native TQ2_0 format (2.06 bpw). No custom SIMD kernels needed -- standard ggml mul_mat handles TQ2_0 transparently.

Quality verification

All 6 variants were tested on JFK audio (11s, 24 kHz). Every variant produces identical output:

ANd so, my fellow Americans, ask not what your country can do for you -- ask what you can do for your country..

Sweep ran on Kaggle (30 GB RAM, CPU). Conversion + transcription took ~20 min total for all 6 variants.

Usage

CLI

# Auto-download (uses the default tq2 variant)
crispasr -m auto --backend vibevoice-bitnet -f audio.wav

# Or with a specific file
crispasr -m vibevoice-asr-bitnet-aggro.gguf --backend vibevoice -f audio.wav

C API

#include "vibevoice.h"

vibevoice_context_params params = vibevoice_context_default_params();
params.n_threads = 4;
vibevoice_context *ctx = vibevoice_init_from_file("vibevoice-asr-bitnet-aggro.gguf", params);
char *text = vibevoice_transcribe(ctx, samples_24khz, n_samples);
printf("%s\n", text);
free(text);
vibevoice_free(ctx);

Python

import crispasr
s = crispasr.Session("vibevoice-asr-bitnet-aggro.gguf", backend="vibevoice")
result = s.transcribe("audio.wav")
print(result["text"])

Conversion

# Default (Q8_0 VAE + F16 embed)
python models/convert-vibevoice-bitnet-to-gguf.py \
    --input microsoft/VibeVoice-ASR-BitNet \
    --output vibevoice-asr-bitnet-tq2.gguf

# Aggressive (Q4_0 VAE + Q8_0 embed, 1.05 GB)
python models/convert-vibevoice-bitnet-to-gguf.py \
    --input microsoft/VibeVoice-ASR-BitNet \
    --output vibevoice-asr-bitnet-aggro.gguf \
    --vae-quant q4_0 --embed-quant q8_0

Acknowledgements

• microsoft/VibeVoice for the model architecture and training
• microsoft/VibeASR.cpp for the BitNet quantization approach

Provenance and EU AI Act Art. 53 note

• Upstream model: microsoft/VibeVoice-ASR-BitNet — published by microsoft.
• Upstream licence: mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.