Parakeet TDT 0.6B v3 — MLX Mixed 4-bit/8-bit
A multilingual mixed-precision MLX conversion of
nvidia/parakeet-tdt-0.6b-v3,
built from the verified unquantized FP32
mlx-community/parakeet-tdt-0.6b-v3
checkpoint.
The 0.401 GB safetensor uses group-64 affine Q4 for 217 encoder matrices and
Q8 for 48 pointwise-convolution plus eight predictor/joint matrices. This is
17.96% smaller than the 0.489 GB
sonic-speech/parakeet-tdt-0.6b-v3-int4
control while retaining its quality class on a complete six-language gate:
11.17% versus 11.21% macro WER. The 0.04-point difference is not presented as
an accuracy improvement; the useful result is practical quality parity with
less weight storage.
Complete multilingual evaluation
Measured locally on Apple Silicon with MLX and public parakeet-mlx
inference. The gate scores every FLEURS test utterance for German, French,
Dutch, Swedish, Danish, and Finnish: 4,509 utterances, 14.64 audio
hours, and zero skipped references for every artifact. Macro WER gives each
language equal weight.
| Model | Weights (GB) | German | French | Dutch | Swedish | Danish | Finnish | Macro WER | RTFx |
|---|---|---|---|---|---|---|---|---|---|
| MLX Community FP32 | 2.508 | 4.56% | 6.18% | 7.00% | 14.38% | 18.02% | 12.09% | 10.37% | 54.7x |
| Sonic encoder-INT4 | 0.489 | 4.70% | 6.59% | 7.32% | 15.58% | 19.60% | 13.44% | 11.21% | 62.1x |
| This mixed Q4/Q8 model | 0.401 | 4.65% | 6.56% | 7.35% | 15.46% | 19.67% | 13.34% | 11.17% | 69.5x |
Against FP32, this model removes 84.02% of weight storage for a 0.80-point macro-WER increase. RTFx includes audio loading, log-mel preprocessing, inference, and decoding, but excludes model loading and dataset preparation. Run order was sequential rather than thermally interleaved, so throughput is directional and should be remeasured on deployment hardware.
Allocation and rejected candidates
Calibration used two disjoint FLEURS validation clips per language (12 clips,
seed 31415). Candidate selection used 25 held-out test rows per language
(150 utterances, seed 42). Only the promoted candidate then ran over all test
rows.
The accepted allocation is:
- Q4: encoder pre-projection and all attention/feed-forward matrices, 217 modules;
- Q8: every pointwise convolution plus predictor and joint matrices, 56 modules;
- BF16: ineligible normalization, residual, bias, and depthwise-convolution tensors.
The selection screen rejected uniform Q4 at 0.354 GB and 10.92% macro WER, and an English-v2-derived Q3/Q4 allocation at 0.367 GB and 11.12%. Isolated low-drift attention groups did not compose safely at Q3 across the multilingual model. Protecting convolution and decoder/head paths at Q8 produced the stronger size/quality point.
Provenance and reproducibility
- NVIDIA source revision:
7c35754d166cca382ad1e53e68b01e7c575f3a1d - MLX Community FP32 revision:
ed2b7e8c15f9aaa0b5772e2efb986255eaef7e15 - Sonic INT4 revision:
aa25511e86a4a83285774ba03df07ca62069de2f
The Sonic artifact was audited directly. Its 217 packed matrices reproduce exactly by casting the pinned FP32 anchor to BF16 and applying MLX affine Q4/group-64; all remaining tensors equal the anchor cast to BF16.
The release was rebuilt from the pinned FP32 source in a clean output directory and reproduced the evaluated weights byte-for-byte:
model.safetensors SHA-256
8ca561e3d2425ebfc934c58c8c905c9717b020bb80f7c7b8abe6ded72e18872e
The exact resolved allocation is included as quantization_recipe.json. The
compatibility loader is included as parakeet_unified_mlx.py; conversion code,
sensitivity results, rejected profiles, and matched benchmark evidence live in
yuting1214/model-lab.
Usage
Install the public runtime and loader dependencies:
pip install parakeet-mlx huggingface_hub pyyaml librosa soundfile
from huggingface_hub import snapshot_download
from parakeet_unified_mlx import load_parakeet_mlx
model_dir = snapshot_download(
"MarkChen1214/parakeet-tdt-0.6b-v3-MLX-Mixed-4bit8bit"
)
model = load_parakeet_mlx(model_dir)
result = model.generate("speech.wav")
print(result.text)
Run the script from the downloaded repository directory, or place
parakeet_unified_mlx.py on PYTHONPATH before importing it.
Scope and limitations
- The upstream model supports 25 European languages, but the local complete
benchmark above covers only
de_de,fr_fr,nl_nl,sv_se,da_dk, andfi_fi. de_dedoes not qualify Austrian or Swiss German, andfr_frdoes not qualify Canadian or Quebec French. Regional product audio is still required.- Measurements use offline full-context TDT inference and greedy decoding; this is not a native Swift or streaming-latency result.
- The included loader is required because current public
parakeet-mlxdoes not instantiate per-module mixed bit widths from the saved recipe itself. - Accuracy varies with accents, domains, noise, and recording conditions. Evaluate on the intended target distribution before deployment.
License
The upstream model is released under CC-BY-4.0. This quantized derivative keeps that license and attribution. See the upstream model card for details.