Parakeet Unified English 0.6B — MLX Mixed 3-bit/4-bit/6-bit
A sensitivity-guided mixed-precision MLX conversion of
nvidia/parakeet-unified-en-0.6b,
built from the full-precision MLX conversion
littlebearlabs/parakeet-unified-en-0.6b-mlx.
The checkpoint uses group-64 affine quantization: 200 eligible matrices at 3-bit, 64 at 4-bit, and nine sensitive frontend, predictor, and joint matrices at 6-bit. Normalization, depthwise convolution, bias, and other ineligible weights remain at their source precision.
At 0.352 GB of safetensor weights, it is 9.89% smaller than the 0.391 GB external uniform-INT4 checkpoint. On the full 6,206-utterance English gate it scores 2.78% macro WER, compared with 2.66% for its FP32 teacher.
Full three-collection evaluation
Measured locally on Apple Silicon with MLX and public parakeet-mlx RNNT
inference. The gate uses the complete LibriSpeech test-clean (2,620), complete
LibriSpeech test-other (2,939), and complete FLEURS en_us test (647) splits.
Every artifact scored all 6,206 utterances, totaling 12.51 audio hours,
with zero skipped references. Macro WER gives each collection equal weight.
| Model | Weights (GB) | LS clean WER | LS other WER | FLEURS en_us WER | Macro WER | RTFx |
|---|---|---|---|---|---|---|
| Unified FP32 MLX teacher | 2.474 | 1.31% | 2.81% | 3.85% | 2.66% | 56.4x |
| This Unified mixed 3/4/6-bit model | 0.352 | 1.36% | 2.90% | 4.07% | 2.78% | 59.8x |
| TDT v2 FP32 MLX | 2.472 | 1.31% | 2.82% | 4.18% | 2.77% | 85.7x |
| TDT v2 mixed 3/4-bit/BF16 | 0.349 | 1.34% | 2.89% | 4.28% | 2.84% | 90.5x |
The useful result is the size/quality point: 85.76% less weight storage than full precision for a 0.12-point macro-WER increase. Among the two compressed releases, Unified is the cautious quality choice by 0.06 point; TDT is the throughput choice, measuring 51.3% faster and 1.04% smaller in this run.
Secondary meeting and financial-speech panel
An earlier seed-2026 comparison sampled up to 100 examples from LibriSpeech
clean, LibriSpeech other, AMI, and Earnings-22. AMI contributed 87 valid rows,
for 387 matched utterances. It remains useful domain evidence, but is not the
headline quality gate.
| Model | Weights (GB) | LS clean | LS other | AMI | Earnings-22 | Macro WER |
|---|---|---|---|---|---|---|
| Full-precision MLX teacher | 2.474 | 1.26% | 2.39% | 11.36% | 10.44% | 6.36% |
| External MLX INT8 | 0.699 | 1.26% | 2.39% | 11.21% | 10.39% | 6.31% |
| External MLX INT4 | 0.391 | 1.26% | 2.50% | 11.52% | 10.12% | 6.35% |
| This mixed 3/4/6-bit model | 0.352 | 1.35% | 2.28% | 11.06% | 10.33% | 6.26% |
The experiment also rejected a uniform 3-bit checkpoint. Although it reached 0.314 GB, its matched fast-screen macro WER increased from 7.09% for the dense teacher to 16.90%. Sensitivity scans identified the RNN-T joint as the largest isolated hotspot; simply lowering every eligible matrix to one bit width was not a viable compression strategy.
Provenance and reproducibility
The NVIDIA source .nemo is pinned at revision
fe53cd885760c96b6a5f51a0bfd362cb4584a98b. Its model config and tokenizer are
byte-identical to the dense MLX teacher. All 989 tensor keys match: 965 tensors
are bit-identical, while the remaining 24 are zero-valued
num_batches_tracked counters represented as FP32 rather than INT64.
The dense MLX source is pinned at revision
9e77fe17aaad040ae27d6eb8fb27e58febe4eb34. The final checkpoint was rebuilt
from that revision in a clean output directory and reproduced the evaluated
weights byte-for-byte:
model.safetensors SHA-256
c0843b8d113c7ba73dcf67472c65be3ed5bcde589ca39f90daefc941a65abbf7
The exact per-module allocation is included as quantization_recipe.json.
The compatibility loader is included as parakeet_unified_mlx.py; conversion
code, rejected profiles, and benchmark evidence are maintained in
yuting1214/model-lab.
Usage
Install the public MLX runtime and the loader dependencies:
pip install parakeet-mlx huggingface_hub pyyaml librosa soundfile
from huggingface_hub import snapshot_download
from parakeet_unified_mlx import load_parakeet_unified
model_dir = snapshot_download(
"MarkChen1214/parakeet-unified-en-0.6b-MLX-Mixed-3bit4bit6bit"
)
model = load_parakeet_unified(model_dir)
result = model.generate("speech.wav")
print(result.text)
Run the script from the downloaded repository directory, or place
parakeet_unified_mlx.py on PYTHONPATH before importing it.
Scope and limitations
- The published quality measurements use offline, full-context RNNT inference.
- The upstream architecture supports offline and buffered streaming modes, but the open runtime used here does not yet reproduce the Unified model's trained dynamic-chunk streaming path. No streaming-latency or streaming-WER claim is made for this checkpoint.
- The external INT4/INT8 cards target a separate runtime. This comparison loads all checkpoints through the same public compatibility path instead.
- Four LSTM matrices are stored at 6-bit but dequantized during load because the current public MLX LSTM implementation has no quantized LSTM kernel.
- Accuracy varies with accents, domains, noise, and recording conditions. Evaluate on your own target distribution before deployment.
License
The original NVIDIA model is released under the NVIDIA Open Model License. This quantized derivative inherits those terms. Review the license for your intended use and retain the required attribution.