mlx-Yue2-3B

This repository provides pre-converted native MLX weights for YuE2-3B, an advanced music foundation model by M-A-P (Multimodal Art Projection).

Complete inference source code, CLI, and documentation are hosted on GitHub: 👉 https://github.com/vanch007/mlx-Yue

Model Description

• Architecture: Native Apple Silicon implementation of YuE2 autoregressive planning (AR), flow-matching acoustic diffusion (NAR), and high-fidelity 48kHz FP32 Oobleck VAE decoding.
• Precision Variants Included:
  • ar-bf16.safetensors: Full-precision BF16 Autoregressive model (4.33 GB).
  • ar-8bit.safetensors: 8-bit quantized Autoregressive linear layers (2.66 GB), reducing memory footprint by ~39% while accelerating generation to ~83 tokens/s on M3 Max.
  • nar-bf16.safetensors: BF16 Non-Autoregressive acoustic flow-matching model (2.93 GB).
  • qwen.tiktoken & config files.

Quick Start with mlx-Yue

1. Install mlx-Yue

Follow the setup instructions in the GitHub Repository:

git clone https://github.com/vanch007/mlx-Yue.git
cd mlx-Yue
uv sync --frozen --no-dev --extra transcription
source .venv/bin/activate

2. Download Weights

Download the complete model weights (including AR, NAR, and the VAE decoder) in one step:

python -c '
from huggingface_hub import snapshot_download
snapshot_download(repo_id="vanch007/mlx-Yue2-3B", local_dir="models/mlx-Yue2-3B")
'

The repository contains both the generator weights and the 48kHz FP32 Oobleck VAE decoder in vae/.

3. Generate Music

Generate a full song using the 8-bit model:

mlx-yue generate examples/full-song.json \
  --model models/converted \
  --vae "$LYRA_VAE" \
  --precision 8bit \
  --output outputs/my_song

Or via Python:

from lyra import YuE2Pipeline

pipe = YuE2Pipeline.from_pretrained(
    "models/mlx-Yue2-3B",
    precision="8bit",
    local_files_only=True
)

result = pipe(
    style="80s synthpop, upbeat energetic drums, female vocal",
    lyrics="""[verse]
City lights neon glow
Running fast losing control
[chorus]
Dance with the stars through the electric night
We are the dreamers chasing the light
"""
)
result.save_artifacts("outputs/my_song")

Benchmarks (Apple M3 Max)

Citation & Acknowledgments

• Source Model: YuE2-3B by Multimodal Art Projection
• MLX Framework: Apple MLX
• mlx-Yue GitHub: https://github.com/vanch007/mlx-Yue