ERes2NetV2 Speaker Verification (ONNX)

This repository provides an optimized, single-file ONNX Opset 18 export of Alibaba's ERes2NetV2 speaker verification model, converted from iic/speech_eres2netv2_sv_zh-cn_16k-common.

All weights are consolidated directly into eres2netv2.onnx (~71 MB) for drop-in deployment without external data blobs.

Model Details

• Architecture: ERes2NetV2
• Parameters: ~17.8 Million
• File: eres2netv2.onnx (single self-contained model, float32)
• Input Feature: 80-dimensional log-Mel filterbanks (16 kHz, 25ms window, 10ms hop, Povey window, Cepstral Mean Normalization)
• Input Tensor: feature with shape [batch_size, frame_num, 80] (dynamic time & batch)
• Output Tensor: embedding with shape [batch_size, 192]
• Normalization: The graph outputs raw embeddings ($L_2$ norm $\approx 126$). You must apply $L_2$ unit normalization (emb / ||emb||) before calculating cosine similarity.

Python Usage (ONNX Runtime)

import numpy as np
import onnxruntime as ort
import soundfile as sf
import torch
import torchaudio

# 1. Load audio and compute 80-dim Kaldi FBanks
audio, sr = sf.read("voice.wav", dtype="float32")
if audio.ndim > 1:
    audio = np.mean(audio, axis=-1)

waveform = torch.from_numpy(audio).unsqueeze(0)
if sr != 16000:
    waveform = torchaudio.transforms.Resample(sr, 16000)(waveform)

fbank = torchaudio.compliance.kaldi.fbank(
    waveform,
    num_mel_bins=80,
    frame_length=25.0,
    frame_shift=10.0,
    dither=0.0,
    sample_frequency=16000,
)
fbank = (fbank - fbank.mean(dim=0, keepdim=True)).unsqueeze(0).numpy().astype(np.float32)

# 2. Extract embedding
session = ort.InferenceSession("eres2netv2.onnx", providers=["CPUExecutionProvider"])
raw_embedding = session.run(["embedding"], {"feature": fbank})[0].squeeze()

# 3. Apply L2 normalization
embedding = raw_embedding / np.linalg.norm(raw_embedding)

Rust Usage (ort + kaldi-native-fbank)

use ndarray::Array3;
use ort::session::Session;

// Run session with shape [1, frame_num, 80]
let outputs = session.run(ort::inputs![
    "feature" => input_tensor.view()
]?)?;

let (_shape, raw_data) = outputs["embedding"]
    .try_extract_raw_tensor::<f32>()?;

let mut embedding = [0.0f32; 192];
embedding.copy_from_slice(raw_data);

// Normalize to unit sphere
let norm = embedding.iter().map(|&x| x * x).sum::<f32>().sqrt();
for x in embedding.iter_mut() {
    *x /= norm;
}

Operating Thresholds

For raw cosine similarity ($A \cdot B$ on unit vectors):

• Same Speaker (Genuine Match): > 0.65
• Ambiguous / Distorted: 0.55 – 0.65
• Different Speaker (Impostor): < 0.55 (studio/gender matched pairs hover around ~0.48 – 0.52)

For Agglomerative Hierarchical Clustering (AHC) in diarization pipelines, use Average Linkage with a cosine distance threshold ($1.0 - \text{cosine}$) of 0.36 – 0.38.