AlphaAvatar Persona - Speaker Vector (ERes2NetV2, ONNX)
Speaker-embedding model used by the AlphaAvatar Persona plugin
(alphaavatar.persona.speaker.vector). It maps an 80-bin Kaldi filterbank
sequence to a 192-dimensional speaker embedding.
This repository replaces the branch-based storage in
AlphaAvatar/plugins-persona
(branch speaker_vector_onnx), which is now legacy.
Artifact
| File | model.onnx |
| SHA256 | a8614dde1e71f5091ce35e031de672d4d87fe1dd839ffe968867febec67e8123 |
| Size | 68.1 MiB |
| ONNX IR / opset | 6 / 11 |
| Exported by | pytorch 2.4.1 |
| Parameters (post-fold) | 17,838,280 |
model.onnx is a byte-identical copy of eres2netv2.onnx from
AlphaAvatar/plugins-persona at commit 8bb7633a3c0116ca68b0de476c65285c7820a1dd (branch
speaker_vector_onnx). Only the file name changed. The SHA256 above equals the
Git-LFS object id recorded on that branch, so the claim is checkable without
downloading both copies. No conversion, re-export or re-quantization was performed.
Provenance (verified, not assumed)
Upstream: ModelScope iic/speech_eres2netv2_sv_zh-cn_16k-common
(pretrained_eres2netv2.ckpt, SHA256 0eb4057106b2573dd7b132cf0c36273ab29afd192c1610f80baa9c556dbb963c),
declared Apache License 2.0 in its model card.
Three ONNX initializers survive export without name mangling. All three are bit-identical to the same-named tensors in that checkpoint:
| tensor | shape | bit-identical |
|---|---|---|
layer3_ds.weight | [1024, 512, 3, 3] | yes |
seg_1.weight | [192, 20480] (3,932,160 values) | yes |
seg_1.bias | [192] | yes |
The other 178 initializers are named onnx::Conv_* because Conv+BatchNorm were
folded at export time, so they cannot be value-matched against raw checkpoint
tensors. Parameter counts agree once folding is accounted for: checkpoint
17,896,745 vs ONNX 17,838,280 (difference 58,465 = folded BatchNorm buffers).
Not established: who ran the export, and with what script. The legacy
repository contains no export script and the ONNX carries no metadata_props.
Recorded as unknown in model_manifest.json rather than guessed.
Input / output
input feature float32 [batch_size, frame_num, 80]
output embedding float32 [batch_size, 192]
Verified with onnxruntime 1.29.0 on CPUExecutionProvider: batch 1 and 2,
frame counts 100 / 298 / 500, deterministic across repeated runs.
Preprocessing
16 kHz mono. Kaldi-style log-mel filterbank, 80 bins, dither 0.0, then per-dimension mean normalisation over time:
from torchaudio.compliance import kaldi as Kaldi
feat = Kaldi.fbank(wav, num_mel_bins=80, sample_frequency=16000, dither=0.0)
feat = feat - feat.mean(0, keepdim=True) # [T, 80]
feat = feat.unsqueeze(0) # [1, T, 80]
AlphaAvatar drives this over a 3.0 s window with a 1.0 s hop
(window_size_samples=48000, step_size_samples=16000).
Postprocessing
The graph output is not L2-normalised (measured L2 norm 125.49 on the example below). Normalise before computing cosine similarity:
emb = emb / np.linalg.norm(emb, axis=-1, keepdims=True)
score = float(a @ b.T)
The upstream ModelScope configuration.json records a same/different decision
threshold of 0.360. AlphaAvatar applies its own SPEAKER_MATCH_THRESHOLD,
which lives in the AlphaAvatar repository and is not part of this artifact.
Examples
examples/input.json describes a deterministic synthetic input (exact numpy
recipe plus the SHA256 of the raw float32 bytes) and examples/output.json
holds the resulting 192-d embedding. The input is synthetic noise, not speech:
it pins the IO contract and numerical determinism, not acoustic behaviour.
Repository files
| file | purpose |
|---|---|
model.onnx | the model (byte-identical to the legacy branch artifact) |
model_manifest.json | authoritative manifest: provenance, verified pre/postprocessing, known issues |
config.json | minimal ONNX descriptor; also the Hub's download-count query file. Not a transformers config — it deliberately carries no model_type/architectures |
examples/ | a deterministic input and the exact outputs it produces |
checksums.sha256 | covers every file above |
Licence
Apache-2.0, inherited from the verified upstream. See LICENSE and NOTICE.