Nemotron 3 Diarization · MLX INT8
Apple Silicon MLX export of NVIDIA Nemotron 3 Diarization. It emits frame-level activity probabilities for up to eight speakers. It does not transcribe speech or identify people.
Model
| Property | Value |
|---|---|
| Parameters | 99.2 million in the source model |
| Weights | INT8 affine, group size 64; other tensors FP16 |
| Format | MLX safetensors |
| File size | About 102 MiB of model weights |
| Audio | 16 kHz mono, 128 mel bands |
| Chunk | 30.4 seconds of mel features; 10 ms output frames |
| Speakers | Up to eight |
Files
| File | Size | Purpose |
|---|---|---|
model.safetensors | About 102 MiB | Quantized weights |
config.json | About 2 KiB | Shapes, cache settings, source revision, and export checks |
mlx_reference.py | About 6 KiB | Low-level MLX inference implementation |
run_chunk.py | About 2 KiB | First-chunk command line example |
LICENSE | About 3 KiB | OpenMDW 1.1 license |
NOTICE | Under 1 KiB | Source and copyright notice |
Performance
Measured on an Apple M5 Pro running macOS 26.6.2 with one 30.4-second chunk from each of three locally held speech recordings. Times are the median of three warm runs of the model stages. Mel extraction, file loading, speaker-cache updates, and segment postprocessing are excluded. The comparison is against the final unquantized checkpoint; this is a numerical parity check, not a diarization error rate benchmark.
| Measure | MLX INT8 | Meaning |
|---|---|---|
| Median model-stage time | 32.7 ms | About 930× faster than the chunk's audio duration; lower is better |
| Mean absolute probability difference | 0.00036–0.00073 | Close to the source checkpoint; lower is better |
| Speaker-activity decision disagreement at 0.25 threshold | 0.01–0.07% of frame/channel pairs | Quantization changes few decisions; lower is better |
The speech-swift host runtime scored 10.98% pooled DER on 132 internal benchmark recordings (12.82 hours) across eight languages, using a 0.25-second collar, overlap included, and automatic speaker count. All files completed. This measures the full Swift audio-to-segment pipeline, not the first-chunk example below. Other hosts must reproduce the source model's cache and FIFO update algorithm to preserve speaker labels across chunks.
Usage
Install mlx, numpy, and safetensors. Supply a NumPy file containing preprocessed 128-band log-mel features with shape [1, 3040, 128], matching the source NeMo preprocessor. The command below processes the first chunk with empty speaker memory:
python run_chunk.py --format mlx --bundle . --mel chunk.npy --output probabilities.npy
import mlx.core as mx
from mlx_reference import Nemotron3MLXReference, load_weights
model = Nemotron3MLXReference(load_weights("model.safetensors"))
# mel: MLX array [1, 3040, 128]; first chunk only
embeddings = model.preencode(mel)
packed = mx.concatenate((embeddings, mx.zeros((1, 304, 512))), axis=1)
probabilities_10ms, probabilities_80ms = model.infer_preencoded(packed, 380)
The first 3040 output frames correspond to the 30.4-second input. Binarize speaker activity with 0.5 onset and offset thresholds, as in the source model's default postprocessor. The pred_score_threshold: 0.25 value in config.json is used to select speaker-cache frames; it is not the segment activity threshold. For later chunks, preserve the arrival-order speaker cache and FIFO state as described by the source model.
Source
Converted from nvidia/Nemotron-3-Diarization, revision a435e9867d79e789e90053f9b6d6834053af564a. NVIDIA's model description explains the architecture and benchmark protocol. The source model and these derived weights are distributed under OpenMDW 1.1; see LICENSE and NOTICE.
Links
- speech-swift — Apple SDK
- Docs — installation and CLI guide
- soniqo.audio — website
- blog — updates