parakeet-tdt-0.6b-v3-mlx-bf16
mlx-community/parakeet-tdt-0.6b-v3 with every floating-point tensor stored as bfloat16
instead of float32. 2.34 GB → 1.25 GB. Nothing else is changed: same architecture, same
vocabulary, same tokenizer.
Why this is not a quantization
Loaders in mlx-audio-swift cast every floating-point
parameter to bfloat16 immediately after reading the checkpoint — ParakeetModel.fromDirectory
defaults to computeDType: .bfloat16 and applies it to all parameters before the first
inference. An fp32 checkpoint therefore ships ~1.1 GB of precision that is discarded at load.
This repo performs that same rounding once, at conversion time, rather than on every load. The values the model computes with are identical.
bfloat16, not float16. bf16 keeps fp32's 8-bit exponent and is exactly the cast the loader performs. fp16 has a 5-bit exponent, so an fp16 checkpoint takes a different, lossier path (fp32 → fp16 → bf16) and does not reproduce fp32 output — measured below.
Measured
Transcribed against the fp32 original and compared word by word. WER is measured against the fp32 output, not ground truth: the question is whether this checkpoint changes anything.
The fp32 baseline was first verified byte-identical across three runs on the same input, so any difference is the checkpoint rather than run-to-run variation.
| Audio | Language | Words | this (bf16) | a public fp16 conversion |
|---|---|---|---|---|
| Talk, 31 min | Danish | 6,551 | 0.00% | 2.79% |
| Talk, 2.5 min | Swedish | 357 | 0.00% | 2.24% |
| Narration, 2.5 min | English | 234 | 0.00% | — |
| Meeting recording | Danish/Swedish | 62 | 0.00% | — |
| Meeting recording | Danish | 39 | 0.00% | — |
| Short clip | Danish | 43 | 0.00% | 0.00% |
Zero edits in ~7,300 words across three languages.
For comparison, on the same audio, 8-bit quantizations of this model measure 0.00% WER on English and 4.8–9.3% on Danish and Swedish — quantization damage that an English-only benchmark does not reveal.
Peak RSS during transcription drops from ~2.6 GB (fp32) to ~1.6 GB.
Conversion
import mlx.core as mx
weights = mx.load("model.safetensors")
out = {k: (v.astype(mx.bfloat16) if v.dtype in (mx.float32, mx.float16) else v)
for k, v in weights.items()}
mx.save_safetensors("model.safetensors", out)
config.json, tokenizer.model, tokenizer.vocab and vocab.txt are copied unmodified.
Attribution
Derived from nvidia/parakeet-tdt-0.6b-v3
via mlx-community/parakeet-tdt-0.6b-v3,
both CC-BY-4.0, and released under the same licence. The only modification is the dtype of the
stored weights, as described above.