Semantic-VAD whisper-tiny
An audio-only end-of-turn detector for voice agents. Give it the last 8 s of the caller's audio at 16 kHz and it returns the probability that the caller has finished speaking. It needs no transcript, so it can answer the moment a pause begins.
It is a Whisper-tiny encoder (8 M parameters) with a small head, shipped as int8 ONNX (10 MB) for one CPU thread and as PyTorch weights. It is the cheapest of the family and the one to run inside an agent process.
Results
The charts below put every model of the family next to the open detectors, at the budgets LiveKit's eot-bench reports: the fewest interruptions a model can reach within 300 ms and 600 ms of average waiting, and the shortest wait it can reach while interrupting at most 5 % and 10 % of pauses.
Every operating point
Each curve is the best trade a policy can reach: how long a finished turn waits against how many pauses get interrupted. Lower left is better.
Per language on eot-bench-data, the new model's int8 export is better than the 2026-09-07 release's in all 14 languages:
What changed on 2026-10-03
This revision replaces the release of 2026-09-07, which is kept on the branch release-2026-09-07. It is better on both test sets, on every
number, in int8 and in PyTorch:
| telephony test: cut-offs at 300 ms | telephony: latency at 5 % | telephony AUC | eot-bench-data: cut-offs at 300 ms | eot-bench-data: latency at 5 % | eot-bench-data AUC | |
|---|---|---|---|---|---|---|
| this model, int8 ONNX | 44.7 % | 1,836 ms | 0.841 | 28.4 % | 896 ms | 0.867 |
| this model, PyTorch | 45.3 % | 1,834 ms | 0.842 | 27.9 % | 894 ms | 0.870 |
| previous release, int8 ONNX | 50.7 % | 2,000 ms | 0.808 | 33.2 % | 919 ms | 0.837 |
| previous release, PyTorch | 50.0 % | 1,983 ms | 0.811 | 32.3 % | 904 ms | 0.846 |
| LiveKit Turn Detector v1 | not run | 14.8 % | 688 ms | 0.941 | ||
| LiveKit Turn Detector v1-mini, audio | 61.6 % | 2,043 ms | 0.748 | 29.9 % | 924 ms | 0.840 |
| SmartTurn v3.2, CPU int8 | 69.6 % | 2,274 ms | 0.670 | 39.0 % | 914 ms | 0.789 |
| Scicom Semantic VAD, large, GPU | 40.7 % | 1,779 ms | 0.861 | 15.8 % | 718 ms | 0.937 |
| silence timer only | 77.6 % | 2,020 ms | 54.9 % | 1,100 ms |
Lower is better for cut-offs and latency, higher for AUC.
- Cut-offs at 300 ms: the share of mid-turn pauses the agent would interrupt if it may wait 300 ms on average after a finished turn. Latency at 5 %: the mean wait after a finished turn if at most 5 % of pauses may be interrupted. AUC at 0.2 s into each pause. All three come from LiveKit's eot-bench: every silence of at least 0.1 s, every policy of threshold, action delay and timeout, the best policy at each budget.
- eot-bench-data: LiveKit's own 14-language set (
livekit/eot-bench-data, validation), scored on the same span set as LiveKit's published runs. - The previous release's telephony numbers are slightly flattering: its checkpoint was chosen by early stopping on a sample of the telephony test split. This model never saw that split before it was scored.
In short, as shipped in int8: 6.0 points fewer interruptions on telephony and 164 ms less waiting there, and 4.8 points fewer on LiveKit's set.
Usage
ONNX on one CPU thread, no torch needed:
import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import WhisperFeatureExtractor
REPO, SR, WINDOW = "Scicom-intl/semantic-vad-eot-whisper-tiny", 16000, 8 * 16000
opts = ort.SessionOptions(); opts.intra_op_num_threads = 1
sess = ort.InferenceSession(hf_hub_download(REPO, "onnx/model.int8.onnx"), opts, providers=["CPUExecutionProvider"])
fe = WhisperFeatureExtractor(feature_size=80, sampling_rate=SR, chunk_length=8)
def p_end_of_turn(pcm: np.ndarray) -> float:
"""pcm: float32 in [-1, 1] at 16 kHz, the caller's audio up to now (any length)."""
pcm = np.asarray(pcm, dtype=np.float32)
if pcm.size and np.abs(pcm).max() > 1.5: # int16-scale samples to unit float
pcm = pcm / 32768.0
pcm = pcm[-WINDOW:] if len(pcm) >= WINDOW else np.pad(pcm, (WINDOW - len(pcm), 0))
feats = fe([pcm], sampling_rate=SR, return_tensors="np", padding="max_length", max_length=WINDOW,
truncation=True, do_normalize=False)["input_features"].astype(np.float32)
return float(sess.run(None, {"input_features": feats})[0].reshape(-1)[0])
Threshold: 0.5. eot-bench's best policies for this model's int8 export use 0.53 to 0.57 at the 300 ms budget, close to the previous release's 0.54 to 0.60.
LiveKit Agents: wrap p_end_of_turn as the predict(pcm) backend of
STT-API's SemanticVAD turn detector. LiveKit hands the
backend int16-scale samples; the snippet rescales them.
To go back to the previous release, pass revision="release-2026-09-07" to hf_hub_download.
Serving cost
About 30 ms per prediction on one CPU thread (int8, 28.5 ms at export, 34 ms in our serving runs). A call asks the detector about 0.15 times a second, so under 1 % of one core per call. A 164-core node sustains about 1,670 predictions a second with 64 single-thread processes, and a 4-core node about 120. The cost does not depend on how much audio you send; the window is always 8 s.
Files
| file | what |
|---|---|
onnx/model.int8.onnx | MatMul-only dynamic int8, 10 MB; probability differs from PyTorch by 0.017 on average, 0.047 at most |
onnx/model.fp32.onnx | fp32 export, 32 MB; within 9e-7 of PyTorch |
onnx/export_report.json | sizes, parity against PyTorch, latency at export |
encoder/ | the fine-tuned encoder, HF format (config.json, model.safetensors) |
eot_head.pt | {"state_dict": LayerNorm, Linear(384, 256), GELU, Linear(256, 1), "pooling": "last5"} |
eot_window.json, preprocessor_config.json | the input contract: 8 s window, 80 mel bins, 16 kHz, no mel normalisation, mean of the last 5 encoder frames |
Input: input_features [batch, 80, 800] float32, the Whisper log-mel of the last 8 s of audio, left-padded with zeros
when shorter, do_normalize=False. Output: probability [batch, 1], already through the sigmoid.
License
Apache-2.0, like the Whisper encoder it fine-tunes.