Nemotron-Labs-Audex-2B — MLX (4-bit)
A 4-bit MLX conversion of NVIDIA's Nemotron-Labs-Audex-2B for Apple Silicon. The language model is quantized to 4-bit (group size 64); the speech decoder and audio encoder are kept in bf16 (small and quality-sensitive).
The LM shrinks from ~4.5 GB to ~1.3 GB and speech synthesis is roughly 2× faster, with no measurable quality loss on my probes (identical text answers; 0% WER on a Whisper round-trip; full speech-to-speech intact). See the bf16 build for the reference weights.
quant.json in this repo records the quantization, so load_model applies it automatically —
no flags needed.
Capabilities (one model)
- Text generation · Text-to-speech · Speech-to-text (ASR) · Speech-to-speech
Usage
Loader/inference code: akashicMarga/mlx-audio-train (models/audex/).
git clone https://github.com/akashicMarga/mlx-audio-train
pip install mlx mlx-lm mlx-whisper soundfile transformers
huggingface-cli download akashicmarga/Nemotron-Labs-Audex-2B-MLX-4bit --local-dir audex_4bit
from models.audex import load_model, s2s_generate
import soundfile as sf
m = load_model("audex_4bit") # 4-bit applied automatically via quant.json
reply, wav = s2s_generate(m, "question.wav")
sf.write("reply.wav", wav, 16000)
print(reply)
Benchmark (single-prompt probes)
| build | LM size | TTS latency | TTS WER (round-trip) | text answer |
|---|---|---|---|---|
| bf16 | 4.50 GB | 2.5 s | 0% | correct |
| 8-bit | 2.39 GB | 1.5 s | 0% | correct |
| 4-bit | 1.27 GB | 1.2 s | 0% | correct |
License & attribution
Derived from nvidia/Nemotron-Labs-Audex-2B; NVIDIA OneWay Noncommercial License
(non-commercial use only). Model and training by NVIDIA
(arXiv:2607.05196); this repo provides only an MLX 4-bit conversion.