Revox Vocoder 1.0

Revox Vocoder 1.0 is a pitch correctness aware neural vocoder designed for high-performance inference at 48 kHz. Its architecture, PC-NSF-Vocos, combines pitch correctness awareness (PC), neural source-filter excitation (NSF), and Vocos-style spectral generation. It converts a log-Mel spectrogram and aligned pitch conditions into a complex spectrum for waveform synthesis. The network has 4,463,874 parameters.

Design principles: PC-NSF-Vocos

The design brings together three techniques:

1. Frequency-domain generation with Vocos and iSTFT. The network predicts Fourier spectral coefficients at the acoustic frame rate, then uses an inverse short-time Fourier transform to synthesize the waveform. This substantially simplifies the learned waveform-synthesis problem: the backbone predicts compact spectral frames, while iSTFT performs the final waveform reconstruction. This division supports efficient inference.

2. NSF excitation inspired by NSF-HiFiGAN. An F0-driven harmonic-plus-noise source supplies an explicit periodic prior. Its projected features guide the spectral generator toward the requested pitch. This inductive bias is introduced to improve convergence speed by giving the network a structured excitation from which to learn voiced audio.

3. Pitch correctness awareness through perturbed input. The conditioning Mel spectrogram is computed from pitch-perturbed audio, while the F0, voicing, validity masks, and waveform target remain at the original pitch. The model is required to reconstruct the original-pitch audio despite the perturbed Mel input. This teaches it to prioritize the supplied F0 when the pitch implied by Mel disagrees with the requested pitch. At inference, Mel can remain fixed while F0 supplies the desired pitch trajectory. This formulation draws on the information-perturbation principle in Neural Analysis and Synthesis (NANSY), applied here specifically to pitch-perturbed Mel conditioning.

Here, PC describes this pitch correctness awareness; NSF describes the internal excitation prior; and Vocos describes spectral prediction followed by iSTFT synthesis. The combination aims to make pitch following an explicit design objective while keeping the inference network compact.

Network architecture

The generator follows this computation:

log-Mel + log-F0 + voicing + pitch validity
                  │
          frame-rate Conv1d ───────────────┐
                                          │
F0 + voicing + pitch validity              │
                  │                       │
       harmonic-plus-noise source         │
                  │                       │
       strided waveform projection ───────┤
                                          │
                                   sum + LayerNorm
                                          │
                                 8 ConvNeXt blocks
                                          │
                                      LayerNorm
                                          │
                                 linear spectral head
                                          │
                                 log-magnitude + phase
                                          │
                                   real + imaginary
                                          │
                                  runtime-side iSTFT
                                          │
                                    48 kHz waveform

Acoustic conditioning

The input contains 128 Mel channels and three pitch-related channels: log-F0, voicing, and pitch validity. Reliable voiced F0 is encoded as ln(max(F0, 1) / 440); other frames use zero for the log-F0 channel. A one-dimensional convolution with kernel size seven projects these 131 channels to 256 channels.

Harmonic source

The internal source generates eight harmonics, weighted by the reciprocal of their harmonic index and normalized by the sum of those weights. Harmonics at or above the Nyquist frequency are discarded. F0 is interpolated between adjacent reliable voiced frames; voicing and validity use the nearest frame center. A frame contributes periodic excitation only when it is voiced, pitch-valid, and has positive F0.

Periodic excitation has gain 0.1. Explicit unit-variance noise is scaled by 0.003 and added to the source. A waveform convolution with a 2,048-sample kernel and stride 480 projects the source to 256 frame-rate channels. These features are added to the acoustic conditioning projection.

The source waveform is encoded by a learned conditioning projection. The network predicts the complete output spectrum rather than multiplying a source spectrum by a filter.

ConvNeXt backbone

Eight residual blocks operate at the Mel frame rate. Each block contains:

1. A depthwise one-dimensional convolution, kernel size seven, 256 channels.
2. Layer normalization.
3. A linear expansion from 256 to 768 channels, GELU, and a linear projection back to 256 channels.
4. A learned per-channel layer scale and residual addition.

Layer normalization also surrounds the block stack. The convolutions use both past and future context.

Spectral head and synthesis

A linear head predicts log-magnitude and phase for 1,025 one-sided frequency bins. Magnitudes are exponentiated after an upper clamp at ln(100). The real and imaginary outputs are magnitude multiplied by the cosine and sine of phase; the imaginary DC and Nyquist bins are zero.

The ONNX graph ends at these real and imaginary spectra. A runtime reconstructs the waveform with a centered inverse STFT using a periodic Hann window, FFT/window size 2,048, hop size 480, and no STFT normalization. Output length is 480 * T samples, or a shorter requested length for the final partial frame. The network does not apply output amplitude clipping.

ONNX interface

File: vocoder.onnx. ONNX opset: 17. Batch size is fixed to 1; frame and source-noise sample counts are dynamic. T must be nonzero.

All feature inputs must share the same frame centers. An unreliable pitch estimate uses pitch_valid=False; it is distinct from a reliable unvoiced decision. Supplying fixed noise makes repeated inference reproducible.

Mel feature convention

Input features correspond to mono audio at 48,000 Hz:

Mel values are ln(max(MelFilterBank @ abs(STFT(audio)), 1e-5)). The centered STFT is truncated to the frame count above. There is no automatic waveform or loudness normalization. Pitch extraction and feature preparation are performed outside the ONNX graph.

Runtime considerations

The network is noncausal. Source phase is continuous within one call and is reset between calls; the graph does not expose streaming state. Independent chunks therefore require runtime handling of context, overlap, and boundaries. Changes in F0 can be supplied per frame while keeping Mel fixed, but large pitch changes or unfamiliar inputs can produce artifacts.

Acknowledgements

This project was inspired by 伊莱莎Ireza.

References

Choi, H.-S., Lee, J., Kim, W., Lee, J., Heo, H., & Lee, K. (2021). Neural analysis and synthesis: Reconstructing speech from self-supervised representations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. Wortman Vaughan (Eds.), Advances in neural information processing systems (Vol. 34, pp. 16251–16265). Curran Associates. https://proceedings.neurips.cc/paper_files/paper/2021/hash/87682805257e619d49b8e0dfdc14affa-Abstract.html

Kong, J., Kim, J., & Bae, J. (2020). HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in neural information processing systems (Vol. 33, pp. 17022–17033). Curran Associates. https://proceedings.neurips.cc/paper_files/paper/2020/hash/c5d736809766d46260d816d8dbc9eb44-Abstract.html

OpenVPI. (n.d.). SingingVocoders [Computer software]. GitHub. https://github.com/openvpi/SingingVocoders

Siuzdak, H. (2024). Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, & Y. Sun (Eds.), International Conference on Learning Representations (pp. 25719–25733). https://proceedings.iclr.cc/paper_files/paper/2024/hash/6db0903efdfe9b1bbafb015c10990b78-Abstract-Conference.html

YuFeng926. (n.d.). OpenTune [Computer software]. GitHub. https://github.com/YuFeng926/OpenTune

Attribution and license

The backbone and spectral head follow the Vocos architecture. Revox adds explicit pitch, voicing, validity, and internal harmonic-source conditioning.

The ONNX weights are licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International. Attribute Minori Live — Revox Vocoder 1.0 and link to this repository when sharing the weights or adaptations.