VocBulwark HiFi-GAN — watermarking neural vocoder (inference export)
Speaker-conditioned BigVGAN / HiFi-GAN neural vocoder. It turns an input mel-spectrogram (what is said) into a 24 kHz waveform, conditioned on a precomputed 768-d speaker embedding (whose voice). Every clip it generates carries a fixed 32-bit provenance watermark for provenance — see Watermark.
This is the lean, inference-only vocoder: the frozen perceptual-loss base models (Whisper / WavLM / Wav2Vec2), the training discriminators, and the speaker encoder have all been stripped — you pass the speaker embedding in. Use the companion speaker-encoder repo to turn a reference clip into that embedding. The modeling code is bundled, so it loads with trust_remote_code=True without the training repo.
For higher watermark capacity (50-bit) and a larger generator (1536 initial channels vs 512), see the large model variant mlr2000/vocoder-large.
Companion Models
This model is part of a set of 6 repositories:
| Repo | Role |
|---|---|
mlr2000/vocoder-large | Large vocoder |
mlr2000/vocoder-large-watermark-detector | Watermark detector for the large model |
mlr2000/vocoder-large-speaker-encoder | Speaker encoder for the large model |
mlr2000/vocoder-small | Small vocoder (this repo) |
mlr2000/vocoder-small-watermark-detector | Watermark detector for the small model |
mlr2000/vocoder-small-speaker-encoder | Speaker encoder for the small model |
Model summary
| Architecture | HiFiGANArchitecture (BigVGAN generator, snakebeta activation) |
| Inputs | log-mel spectrogram (96 mel channels) + speaker embedding (768-d) |
| Output | mono waveform, 24 kHz |
| Speaker conditioning | precomputed embedding (from the companion speaker encoder) |
| Generator | initial channels 512, upsample rates [4, 4, 2, 2, 2, 2] |
| Watermark | 32-bit fixed VocBulwark signature, always embedded |
| Framework | 🤗 Transformers, PyTorch, safetensors |
Usage
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("mlr2000/vocoder-small", trust_remote_code=True).eval()
mel = torch.randn(1, model.config.hifigan_in_channels, 200) # [B, mel, T]
emb = torch.randn(1, model.config.speaker_embedding_size) # [B, 768] from the speaker encoder
with torch.no_grad():
audio = model(mel_spectrogram=mel, speaker_embedding=emb).audio
# audio: [B, 1, samples] @ model.config.target_sample_rate
See example_roundtrip.ipynb in this repo for the full pipeline (reference
clip → speaker encoder → embedding → vocode → verify watermark).
Training
| Training data | Multilingual LibriSpeech (8 languages, ~22,200h) and Common Voice (14 languages, ~3,000h) |
| Training steps | 1,000,000 |
| Hardware | 2 × NVIDIA H100 PCIe GPUs |
| Training objective | Discriminator-free: mel spectrogram + WavLM + wav2vec 2.0 + Whisper encoder losses |
| Effective batch size | 32 |
| Learning rate | 1e-3 |
Watermark
Every clip this model generates carries a fixed 32-bit provenance watermark (config.fixed_watermark). It is embedded automatically inside forward and cannot be disabled or changed through this interface — there is deliberately no watermark argument to override. To check whether a given audio came from this model, use the companion detector repo, which extracts the bits and compares them to the same fixed code.
To verify whether a given audio clip was generated by this model, use the companion detector repo (mlr2000/vocoder-small-watermark-detector), which extracts the embedded bits and compares them to the known fixed code.
Notes
- Inputs: log-mel spectrogram (
config.hifigan_in_channelschannels) and a[B, config.speaker_embedding_size]speaker embedding. - Output: mono waveform at
config.target_sample_rate. - Use the speaker embedding from the speaker encoder this vocoder was trained with. A mismatched encoder will not condition it correctly.
- Not intended for voice cloning of real individuals without consent, or any deceptive or impersonation use.
Citation
If you use this model, please cite:
@misc{muletta2026,
title = {Training a Discriminator-Free Foundation Vocoder
with Integrated Audio Watermarking},
author = {Muletta, Romolo and Deriu, Jan},
year = {2026},
note = {VT2 Project Report, ZHAW School of Engineering}
}
License
cc-by-4.0. Trained on MLS (CC-BY-4.0) and Common Voice (CC0); builds on
BigVGAN (MIT) and wav2vec 2.0 (Apache-2.0). Please retain attribution when
redistributing or building on this model.