MuLaCover — Cover Song & Music Remix Generation
MuLaCover is a controllable AI cover-song and music-remix model. It transforms reference music using lyrics and a structured text style prompt, or generates from melody/chord MIDI. The standalone inference code is maintained in the MuLaCover repository.
HeartMuLa repository · HeartMuLa paper
Model weights are publicly available for download. No access request is required.
Inputs
Lyrics use section markers and preserved line breaks:
[Verse]
One lyric line
Another lyric line
[Chorus]
One chorus line
Style is one UTF-8 line using named fields:
topic:[Longing]; genre:[country]; instrument:[Strings,acoustic guitar]; mood:[hopeful]
Download
Download the model weights with the Hugging Face CLI. Authentication is not required:
hf download HeartMuLa/MuLaCover --local-dir ./ckpt/MuLaCover
Also download HeartCodec and Qwen3-Embedding-0.6B into sibling checkpoint directories. Reference-audio conditioning also needs the documented YourMT3 and ChordNet checkpoints. See the code repository for the complete installation and generation commands.
Usage
Install the mulacover package and prepare all checkpoint directories using
the generation guide.
Run the following from the code repository root. ./ckpt is the parent
directory containing MuLaCover, HeartCodec, Qwen3, and the transcription models.
import torch
from mulacover import MuLaCoverGenPipeline
torch.manual_seed(42)
pipe = MuLaCoverGenPipeline.from_pretrained(
"./ckpt",
device=torch.device("cuda:0"),
dtype={
"mulacover": torch.bfloat16,
"codec": torch.float32,
"qwen": torch.float32,
"transcriptor": torch.float32,
},
lazy_load=True,
)
pipe(
{
"ref_audio": "/path/to/reference.mp3",
"lyrics": "assets/lyrics.txt",
"tags": "assets/tags.txt",
},
save_path="cover.wav",
)
The pipeline writes a WAV file. Use the MuLaCover package API shown above; this model is not a built-in Transformers model class.
Files
config.json: MuLaCover model architecture configuration.gen_config.json: audio/text token identifiers used during generation.model-00001-of-00005.safetensors…model-00005-of-00005.safetensors: sharded model weights.model.safetensors.index.json: shard index.tokenizer.json: lyrics and prompt tokenizer.SHA256SUMS: release integrity checksums.
License and use restrictions
The official model weights are licensed under CC BY-NC 4.0 with the additional
terms in MODEL_LICENSE. Commercial use of the official weights is prohibited
without separate written authorization from MuLa Labs.
Outputs generated using the official weights are restricted to noncommercial use unless separately authorized in writing by MuLa Labs. The Apache-2.0 license covering the source-code repository does not grant commercial rights to these weights or their generated outputs.
Users are responsible for obtaining the rights required for their lyrics, reference audio, performances, voices, and generated works. Do not use the model to impersonate a person deceptively or violate copyright, privacy, or publicity rights.
Community
Join the MuLaCover Discord for discussion and community support. Reproducible software defects should be reported through the code repository's GitHub Issues.
Join MuLa Labs, Vera Praxis Lab
We are always excited to meet people with a strong interest in audio and music. MuLa Labs has internship openings for candidates who want to build the next generation of music and audio technology. To apply, email your resume or CV, together with a short introduction, to contact@mulalabs.ai.