Realtime RE-USE — Whisper Finetune
A fine-tuned version of NVIDIA Real-time RE-USE optimised for enhancing whispered speech and noisy recordings in ASR pipelines.
Usage
import torch
from Realtime_REUSE.models.streaming_generator_SEMamba_time_d1_random_layer_ahead_sep_conv import (
SEMamba_decoder_list,
)
model = SEMamba_decoder_list.from_pretrained("{repo_id}")
model.eval()
Or via the CLI:
python -m scripts.inference_realtime_reuse \
--input ./noisy_audio/ \
--output ./enhanced_audio/ \
--checkpoint {repo_id}/pytorch_model.bin
Model Architecture
| Property | Value |
|---|---|
| Base model | NVIDIA Real-time RE-USE (SEMamba, 12 layers, 3.7 M params) |
| Architecture | Conv encoder + Mamba blocks + Conv decoders |
| Fine-tuning data | Whisper-format speech (normal + whispered + background noise) |
Training Details
- Base checkpoint: NVIDIA Real-time RE-USE
USEMamba_12x1_lr_00002_norm_05_vq_067_nfft_320_hop_160_NRIR_012_pha_0005_com_04_early_005_release_random_layer_GAN_longer_1k - Optimiser: Optuna-tuned learning rate, early stopping
- Exit layer: 8 (configurable at inference time)
Input / Output
- Input: Mono-channel audio, 8 kHz – 48 kHz (.wav, .flac)
- Output: Enhanced mono-channel audio (.flac)
License
This model is derived from NVIDIA Real-time RE-USE, released under the NVIDIA One-Way Noncommercial License (NSCLv1).