Parakeet TDT 0.6B v2 — MLX Mixed 3-bit/4-bit/BF16
A sensitivity-guided mixed-precision MLX conversion of the English-only
nvidia/parakeet-tdt-0.6b-v2,
built from the unquantized FP32
mlx-community/parakeet-tdt-0.6b-v2
checkpoint.
The 0.349 GB safetensor uses group-64 affine quantization for 265 eligible matrices: 100 attention matrices at 3-bit and 165 encoder, feed-forward, and pointwise-convolution matrices at 4-bit. The eight predictor and joint matrices remain BF16, as do normalization, depthwise-convolution, bias, and other ineligible weights.
It is 25.8% smaller than the 0.470 GB external encoder-INT4 model while matching its macro WER on a secondary 387-row domain panel. On the full 6,206-utterance English gate, it is the throughput-focused option: 51.3% faster and 1.04% smaller than the separately published Unified mixed-precision model, with 0.06 point higher macro WER.
Full three-collection evaluation
Measured locally on Apple Silicon with MLX and public parakeet-mlx inference.
The gate uses the complete LibriSpeech test-clean (2,620), complete
LibriSpeech test-other (2,939), and complete FLEURS en_us test (647) splits.
Every artifact scored all 6,206 utterances, totaling 12.51 audio hours,
with zero skipped references. Macro WER gives each collection equal weight.
| Model | Weights (GB) | LS clean WER | LS other WER | FLEURS en_us WER | Macro WER | RTFx |
|---|---|---|---|---|---|---|
| Unified FP32 MLX | 2.474 | 1.31% | 2.81% | 3.85% | 2.66% | 56.4x |
| Unified mixed 3/4/6-bit | 0.352 | 1.36% | 2.90% | 4.07% | 2.78% | 59.8x |
| TDT v2 FP32 MLX | 2.472 | 1.31% | 2.82% | 4.18% | 2.77% | 85.7x |
| This TDT v2 mixed 3/4-bit/BF16 model | 0.349 | 1.34% | 2.89% | 4.28% | 2.84% | 90.5x |
Against FP32, this model trades 0.07 macro-WER point for an 85.90% reduction in weight storage. There is no universal winner among the compressed models: Unified has the cautious quality edge, while this TDT release is the throughput choice.
Secondary meeting and financial-speech panel
An earlier seed-2026 comparison sampled up to 100 examples from LibriSpeech
clean, LibriSpeech other, AMI, and Earnings-22. AMI contributed 87 valid rows,
for 387 matched utterances. It remains useful domain evidence, but is not the
headline quality gate.
| Model | Weights (GB) | LS clean | LS other | AMI | Earnings-22 | Macro WER | RTFx |
|---|---|---|---|---|---|---|---|
| TDT v2 FP32 MLX | 2.472 | 1.26% | 2.61% | 9.24% | 10.70% | 5.95% | 76.9x |
| External TDT v2 encoder-INT4 | 0.470 | 1.48% | 2.39% | 9.75% | 10.81% | 6.11% | 85.1x |
| This TDT v2 mixed 3/4-bit/BF16 model | 0.349 | 1.57% | 2.67% | 9.54% | 10.61% | 6.10% | 90.3x |
| Repo Parakeet Unified mixed 3/4/6-bit | 0.352 | 1.35% | 2.28% | 11.06% | 10.33% | 6.26% | 55.3x |
The TDT candidate and external INT4 model are a practical quality tie on this panel. Their 0.01-point difference is not an accuracy claim; the useful result is retaining that quality class with 0.121 GB less weight storage.
Allocation and rejected candidates
The calibration set contains 12 FLEURS en_us validation clips selected with
seed 31415; it is disjoint from the final test rows. Four-layer attention,
feed-forward, and pointwise-convolution regions plus the predictor and joint
were independently fake-quantized against a BF16 teacher.
The accepted allocation is:
- encoder attention in layers 0–15 and 20–23: affine Q3;
- encoder attention in layers 16–19: affine Q4;
- all encoder feed-forward matrices: affine Q4;
- encoder pre-projection and all pointwise convolutions: affine Q4;
- predictor embedding, four LSTM matrices, and three joint matrices: BF16.
The broader screen rejected several attractive-looking size points. Uniform Q3 reached 0.272 GB but 11.88% macro WER on the seed-42 screen. Uniform Q4 reached 0.349 GB but 8.17%. A 0.323 GB Q3-attention/Q3-FFN composition reached 8.35%, showing that low isolated calibration drift did not always compose safely.
Provenance and reproducibility
- NVIDIA source revision:
ae9ad07059c7c739ffaf932226a8fe64ae2620b0 - MLX Community FP32 revision:
8ae155301e23d820d82aa60d24817c900e69e487 - External INT4 revision:
35bbe15bda247cada6f3e54b9156cfddf0ad7b51
The final checkpoint was rebuilt from the pinned FP32 source in a clean output directory and reproduced the evaluated weights byte-for-byte:
model.safetensors SHA-256
278c556f6be491dcc1c2972de1ab2f20ae22f6ee47caec474154a9d607c3500c
The exact allocation is included as quantization_recipe.json. The compatibility
loader is included as parakeet_unified_mlx.py; conversion code, sensitivity
results, rejected profiles, and matched benchmark evidence are maintained in
yuting1214/model-lab.
Usage
Install the public runtime and loader dependencies:
pip install parakeet-mlx huggingface_hub pyyaml librosa soundfile
from huggingface_hub import snapshot_download
from parakeet_unified_mlx import load_parakeet_mlx
model_dir = snapshot_download(
"MarkChen1214/parakeet-tdt-0.6b-v2-MLX-Mixed-3bit4bitBF16"
)
model = load_parakeet_mlx(model_dir)
result = model.generate("speech.wav")
print(result.text)
Run the script from the downloaded repository directory, or place
parakeet_unified_mlx.py on PYTHONPATH before importing it.
Scope and limitations
- This is an English-only model. It is not a replacement for multilingual ASR.
- Measurements use offline full-context TDT inference and greedy decoding.
- The included loader is required because current public
parakeet-mlxdoes not instantiate per-module mixed bit widths from this saved recipe by itself. - RTFx is machine-, runtime-, and thermal-state dependent. Treat the matched local comparison as directional and remeasure on your deployment hardware.
- Accuracy varies with accents, domains, noise, and recording conditions. Evaluate on your own target distribution before deployment.
License
The upstream model is released under CC-BY-4.0. This quantized derivative keeps that license and attribution. See the upstream model card for details.