WavLM-large + ECAPA-TDNN speaker verification

UniSpeech's WavLM-large + ECAPA-TDNN speaker-verification model, adapted to use with only torch, torchaudio, and transformers.

import soundfile as sf
import torch
from transformers import AutoModel

device = "cuda" if torch.cuda.is_available() else "cpu"
reference, sr = sf.read("reference.wav", dtype="float32")
candidate, candidate_sr = sf.read("candidate.wav", dtype="float32")
assert candidate_sr == sr

waves = [torch.from_numpy(reference), torch.from_numpy(candidate)]
lengths = torch.tensor([wave.numel() for wave in waves], device=device)
waves = torch.nn.utils.rnn.pad_sequence(waves, batch_first=True).to(device)
model = AutoModel.from_pretrained(
    "prj-beatrice/unispeech-wavlm-large-ecapa-tdnn-torch-native",
    trust_remote_code=True,
).eval().to(device)

with torch.inference_mode():
    embeddings = model(
        waves, sampling_rate=sr, input_lengths=lengths
    ).embeddings
    similarity = torch.nn.functional.cosine_similarity(
        embeddings[:1], embeddings[1:]
    )[0]
print(similarity)

The value matches Seed-TTS Eval's speaker-verification code:

from verification import verification

expected, _ = verification(
    "wavlm_large",
    "reference.wav",
    "candidate.wav",
    checkpoint="wavlm_large_finetune.pth",
    use_gpu=torch.cuda.is_available(),
    device=device,
)
torch.testing.assert_close(similarity.cpu(), expected[0].cpu())

License

The complete model repository is distributed under CC BY-SA 3.0 Unported; see LICENSE.