Frame-VAD Multilingual MarbleNet v2.0
NVIDIA NeMo's 91.5K-parameter frame-level voice activity detector, exported for loom.cpp. Family 13: audio in, a speech probability for every 20 ms frame out.
This is a loom.cpp export: a single self-describing GGUF that carries its own graph topologies, tokenizer (if any) and driver script, produced by loom-exporter.
Original model
Exported from nvidia/frame_vad_multilingual_marblenet_v2.0. Weights are unmodified; this repo packages the same parameters into
loom.cpp's GGUF format.
License
NVIDIA Open Model License Agreement -- inherited from the base model above.
Language(s)
en, es, fr, de, ru, zh
Usage
Run it with loom-py -- loom-py-rt on PyPI:
pip install -U "loom-py-rt[hub]"
import loom
model = loom.Model.from_pretrained("loom-ai-org/marblenet-vad-v2-loom")
# Audio is a mono float list at 16 kHz. One call over the whole clip, one row per 20 ms frame back.
result = model.speech2class.infer(audio)
print(result.granularity, result.labels)
# frame ['non_speech', 'speech']
# A probability per frame, not a decision: the threshold and any smoothing are yours. With a plain
# 0.5 threshold, the speech curve becomes segments:
speech = result.probability("speech")
segments, start = [], None
for t, p in zip(result.times, speech):
if p >= 0.5 and start is None:
start = t
elif p < 0.5 and start is not None:
segments.append((round(start, 2), round(t, 2)))
start = None
if start is not None:
segments.append((round(start, 2), round(len(audio) / 16000, 2)))
print(segments) # [(start_seconds, end_seconds), ...]
The layer underneath
The call above is the high-level door: one per task, named for the modality pair it maps between, with
the windowing, sampling and assembly this model needs already applied. Under it, model.infer(...)
passes your arguments straight to the driver this GGUF embeds -- which is where you go for a knob the
door does not name.
model.driver_source prints that driver, including a header comment documenting every argument it
accepts for this model, and is the authority on it. See loom-py for the API and
loom.cpp for what the engine does between the two.
Known limitations
The answer is a probability per frame, not a decision. Upstream's own card says the output may need smoothing and thresholding (0.5 is its example); both are left to you, because every consumer of a VAD -- an ASR front end, a recorder, a diarizer -- wants different onset and offset behaviour.
One row per 20 ms of input, starting at 0 s: result.times gives each row's start. Audio must be mono 16 kHz.
Distributed under the NVIDIA Open Model License Agreement, which upstream names; read it before shipping this in a product.
Files
marblenet-vad-v2.gguf-- the model, exported with loom-exporter.