VocBulwark HiFi-GAN — watermarking neural vocoder (inference export)
Speaker-conditioned BigVGAN / HiFi-GAN neural vocoder. It turns an input mel-spectrogram (what is said) into a 24 kHz waveform, conditioned on a precomputed 768-d speaker embedding (whose voice). Every clip it generates carries a fixed 50-bit provenance watermark that identifies this specific model instance, see Watermark.
This is the lean, inference-only vocoder: the frozen perceptual-loss base models (Whisper / WavLM / Wav2Vec2), the training discriminators, and the speaker encoder have all been stripped, you pass the speaker embedding in. Use the companion speaker-encoder repo to turn a reference clip into that embedding. The modeling code is bundled, so it loads with trust_remote_code=True without the training repo.
Model summary
| Architecture | HiFiGANArchitecture (BigVGAN generator, snakebeta activation) |
| Inputs | log-mel spectrogram (96 mel channels) + speaker embedding (768-d) |
| Output | mono waveform, 24 kHz |
| Speaker conditioning | precomputed embedding (from the companion speaker encoder) |
| Generator | initial channels 1536, upsample rates [4, 4, 2, 2, 2, 2] |
| Watermark | 50-bit fixed VocBulwark signature, always embedded |
| Framework | 🤗 Transformers, PyTorch, safetensors |
Companion Models
This model is part of a set of 6 repositories:
| Repo | Role |
|---|---|
mlr2000/vocoder-large | Large vocoder (this repo) |
mlr2000/vocoder-large-watermark-detector | Watermark detector for the large model |
mlr2000/vocoder-large-speaker-encoder | Speaker encoder for the large model |
mlr2000/vocoder-small | Small vocoder |
mlr2000/vocoder-small-watermark-detector | Watermark detector for the small model |
mlr2000/vocoder-small-speaker-encoder | Speaker encoder for the small model |
Usage
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("mlr2000/vocoder-large", trust_remote_code=True).eval()
mel = torch.randn(1, model.config.hifigan_in_channels, 200) # [B, mel, T]
emb = torch.randn(1, model.config.speaker_embedding_size) # [B, 768] from the speaker encoder
with torch.no_grad():
audio = model(mel_spectrogram=mel, speaker_embedding=emb).audio
# audio: [B, 1, samples] @ model.config.target_sample_rate
See example_roundtrip.ipynb in this repo for the full pipeline (reference
clip → speaker encoder → embedding → vocode → verify watermark).
Training
| Training data | Multilingual LibriSpeech (8 languages, ~22,200h) and Common Voice (14 languages, ~3,000h) |
| Training steps | 1,000,000 |
| Hardware | 2 × NVIDIA H200 GPUs |
| Training objective | Discriminator-free: mel spectrogram + WavLM + wav2vec 2.0 + Whisper encoder losses |
| Effective batch size | 32 |
| Learning rate | 1e-4 |
Watermark
Every clip this model generates carries a fixed 50-bit provenance watermark (config.fixed_watermark) that identifies this specific model instance. It is embedded automatically inside forward and cannot be disabled or changed through this interface — there is deliberately no watermark argument to override.
To verify whether a given audio clip was generated by this model, use the companion detector repo (mlr2000/vocoder-large-watermark-detector), which extracts the embedded bits and compares them to the known fixed code.
Notes
- Inputs: log-mel spectrogram (
config.hifigan_in_channelschannels) and a[B, config.speaker_embedding_size]speaker embedding. - Output: mono waveform at
config.target_sample_rate. - Use the embedding from the speaker encoder this vocoder was trained with, a mismatched encoder will not condition it correctly.
- Not intended for voice cloning of real individuals without consent, or any deceptive / impersonation use.
Citation
If you use this model, please cite:
@misc{muletta2026,
title = {Training a Discriminator-Free Foundation Vocoder
with Integrated Audio Watermarking},
author = {Muletta, Romolo and Deriu, Jan},
year = {2026},
note = {VT2 Project Report, ZHAW School of Engineering}
}
License
cc-by-4.0. Trained on MLS (CC-BY-4.0) and Common Voice (CC0); builds on
BigVGAN (MIT) and wav2vec 2.0 (Apache-2.0). Please retain attribution when
redistributing or building on this model.