Humaneness Voice Converter

LAION's recommended default Chatterbox-based voice-conversion checkpoint. This is the complete, score-conditioned release; the original Chatterbox checkpoint remains available for comparison and backward compatibility.

Research voice conversion for speech and dataset augmentation. Give the model a spoken source recording and a short recording of a different, consenting target speaker. It attempts to say the source content in the target voice while retaining much of the source performance. It is an audio-to-audio converter, not a text-to-speech model and not the Humaneness Voice Small semantic TTS.

This release is a complete inference checkpoint, not an adapter that asks you to find and merge several LoRAs. It includes the Chatterbox S3 speech-token extractor, the CAMPPlus speaker encoder, the score-conditioned flow decoder, the HiFT vocoder, and the frozen Humaneness Ears Medium score predictor needed by the provided raw-audio inference command. The model itself is based on Resemble AI's Chatterbox; the voice-conversion architecture and training were developed by LAION. The upstream Chatterbox components retain their MIT attribution; see NOTICE.md.

Listen and compare

The Humaneness Voice Converter Demo uses the same 40 curated source recordings and two fixed target voices for untouched Chatterbox, Humaneness Voice Converter, best-of-N selections and separately validated external voice converters. The refreshed set has 19 real recordings, 21 German clips and 34 emotion labels. Unsafe, incoherent and trivial sources were replaced from the prepared, licensed pool; some emotions appear twice instead of forcing a weak example into every class. The demo states the exact subset, evaluation protocol and limitations. It is not a blinded listening study.

The directly measured proxy scores below are single-sample outputs on the 40 curated sources × two targets. Higher AudioBox Content Enjoyment (CE), AudioBox Production Quality (PQ), and target-speaker CAMPPlus cosine are directionally better; lower source-top-two-emotion MSE is better. CE/PQ/emotion are outputs of Humaneness Ears Medium, the same judge family used in training. The sources were curated using input quality, and 10 source UIDs overlap the known RL source list. Speaker-level separation has not been proven.

On this listening set, Humaneness Voice Converter delivers higher predicted Content Enjoyment (+0.284) and Production Quality (+0.243) than Vanilla, while matching the source's strongest emotions more closely (MSE 0.113 to 0.092). That gain comes with a trade-off in target-speaker consistency: the CAMPPlus cosine is 0.773 rather than Vanilla's 0.821. Optional Best-of-N selection improves the combined reward, emotion match and speaker cosine; CE/PQ are already high at one take and change little as N grows. In a separate 93-source, training-UID-disjoint comparison to the 24k predecessor, 40k scored CE 6.074 versus 5.930 and PQ 7.876 versus 7.790; emotion MSE worsened from 0.289 to 0.320 and speaker cosine changed from 0.730 to 0.738. The set is not proven speaker-disjoint. There is currently no independent human MOS or WER result for this release.

How it works

1. Chatterbox extracts roughly 25 S3 semantic tokens per second from the source. These carry speech content and timing without using text.
2. The target supplies a CAMPPlus speaker embedding and a short acoustic prefix. In the tested command the target is 5–10 seconds long. The acoustic prefix and speaker embedding are essential; this is not a text-only voice-cloning system.
3. Frozen Humaneness Ears Medium predicts 99 conditioning scores: 40 emotions, 57 VoiceNet attributes, genuineness and vocal-burst blending. Time-varying style/emotion axes come from the source; selected relatively stable timbre/age/gender axes come from the target. A two-layer score projection conditions the Chatterbox S3Gen flow generator alongside the speaker vector.
4. The score-conditioned flow decoder generates mel features in 10 flow steps, and frozen HiFT renders a 24-kHz waveform.

The model is useful for experimenting with expressive conversions, but a converted voice is not a verified speaker identity. The speaker embedding similarity metric is only a proxy. English and German are represented in the training/evaluation material. Informal tests suggest the pipeline can process other languages, but this release makes no multilingual quality guarantee.

Quick start: two audio files in, one WAV out

Use a CUDA-enabled PyTorch installation. The command below was tested with Python 3.13.5, PyTorch 2.9.1+cu128, Chatterbox 0.1.7 and Transformers 5.14.1. The tested Jupiter environment overrides several upstream Chatterbox 0.1.7 metadata pins. The commands below record that arrangement; they have been validated for inference in the existing Jupiter environment, not as a fresh cross-platform resolver test. Install platform-compatible CUDA Torch first and check versions before using the pinned snapshot.

python -m pip install -r requirements.txt
python -m pip install --no-deps chatterbox-tts==0.1.7
hf download laion/Humaneness-Voice-Converter --local-dir humaneness-vc
python humaneness-vc/infer.py \
  --model-dir humaneness-vc \
  --source source_speech.wav \
  --target consenting_target_voice.wav \
  --output converted.wav \
  --seed 20261011 --flow-steps 10

To draw and rank eight candidate conversions on one GPU, reuse the same source tokens and reference in true GPU microbatches:

python humaneness-vc/infer.py \
  --model-dir humaneness-vc --source source_speech.wav \
  --target consenting_target_voice.wav --output selected.wav \
  --candidates 8 --microbatch 8 --precision bf16 \
  --metrics selected.metrics.json

The ranker uses the included fixed calibration: one third standardized AudioBox Content Enjoyment, one third target CAMPPlus cosine, and one third negative source-top-two-emotion MSE. It is a proxy, not a human judge. --candidates 1 remains the default and does not run candidate ranking. See SPEED.md for measured GH200 FP32/BF16 timings, the Best-of-4/8/16 quality–latency trade-off and what batching excludes.

The input can be WAV or a format supported by your SoundFile build. The validated path accepts finite mono/mixed-down audio of 1–30 seconds; use a clean 5–10-second target with one speaker for comparable results. The provided command was actually run on a GPU using a 6.528-second source and a 5.448-second target, producing a 6.52-second, 24-kHz WAV. A fixed seed makes the stochastic flow sampling repeatable on a fixed software/hardware stack. Different seeds produce different takes; selecting from several takes is an optional, more expensive inference strategy, not part of the single-sample numbers above.

model.safetensors is the complete merged Chatterbox S3Gen state with our 40k-trained flow and 99-score projection. conditioning/ears-medium/ contains the frozen score predictor and its normalization metadata. infer.py is the tested raw-audio command. training/ contains the source snapshots used for the supervised, LoRA/RL and full-flow stages. Details, data selection, parameters, Slurm context, evaluation, caveats and hashes are in REPRODUCING.md. The training scripts retain some Jupiter Scratch path defaults and need an equivalent prepared data layout; they are research source snapshots, not a universal one-command trainer. The demo's refreshed source selection and packed-audio build are documented in research/curate_demo_v2.py and research/build_and_publish_demo_v2.py; the Space manifest lists every public case ID and its source attribution.

Training in brief

The LAION sequence was: a 300.017-hour rank-128 supervised score-conditioned Chatterbox pilot; group-relative flow-RL adapters; a 24,000-group Content-Enjoyment/emotion/speaker "Speaker-Mix" adapter; then merging that adapter and training all 112,620,864 flow-generator parameters for 40,000 more groups of 16. The upstream tokenizer, CAMPPlus, HiFT and Ears judge remained frozen. The 40k reward standardized each component within each 16-candidate group, weighting AudioBox Content Enjoyment 40%, lower error on the source's two strongest emotion axes 30%, and target-speaker CAMPPlus cosine 30%. Training used a 2,000-group warm-up, then a single cosine decay from peak 3e-7 to floor 3e-8, without learning-rate restarts across Slurm segments. Optimizer-bearing checkpoints were saved every 1,000 groups. This is a GROW-inspired group-relative continuous-flow objective, not ordinary discrete-token GRPO. See the code and method notes.

Intended use, limitations and ethics

Primary intended uses are research, comparative evaluation and dataset augmentation with audio and target voices you are authorized to use. Obtain informed consent where appropriate, disclose converted speech, and do not use this model for impersonation, fraud, deception or identity spoofing. Those are ethical-use recommendations, not additional legal restrictions on the CC BY 4.0 license. Independently review voice rights, dataset terms, jurisdictional law and safety controls before deployment. Test on your own languages, microphones, demographics and speaker conditions before any production use. Generated speech can contain artifacts, content changes, unexpected emotion, or identity drift. LAION provides this research release without a promise of fitness, accuracy or safety and is not responsible for downstream uses to the extent permitted by applicable law.

LAION-authored additions and documentation are provided under CC BY 4.0. The bundled Chatterbox-derived components retain the upstream MIT license and attribution; source recordings in external demos keep their own rights. External benchmark models and their outputs are not redistributed as weights in this repository and retain their separate licenses.