parakeet-tdt_ctc-0.6b-ja-bf16

bfloat16 pre-cast of mlx-community/parakeet-tdt_ctc-0.6b-ja (Japanese ASR, Hybrid FastConformer TDT-CTC 0.6B), optimized for on-device inference on iPhone and Apple Silicon Macs.

~1.25 GB — compared to ~2.49 GB for the fp32 upstream conversion.

Why bf16 (modification notice)

MLX runtimes (e.g. mlx-audio-swift ParakeetModel) run this model in bfloat16 by default. Loading the fp32 checkpoint forces an fp32→bf16 cast at load time, during which both copies coexist in memory (peak ≈ 3.7 GB) — a real problem on 8 GB iPhones. With this pre-cast checkpoint the cast is a no-op: download size and load-time peak memory are both halved, with no change in inference behavior.

• Converted with MLX (mlx.core): all float32 / float16 tensors cast to bfloat16; non-float tensors passed through unchanged.
• No re-quantization, pruning, or weight modification beyond the dtype cast.
• Files: config.json (unchanged; vocabulary is embedded), model.safetensors (bf16), vocab.txt (unchanged copy for drop-in compatibility).

Attribution

The original parakeet-tdt_ctc-0.6b-ja model is developed by NVIDIA (trained on ReazonSpeech) and released under CC-BY-4.0. The MLX conversion this checkpoint derives from is by mlx-community. This repository redistributes a bf16 derivative under the same CC-BY-4.0 license.

Usage (mlx-audio-swift)

let model = try await ParakeetModel.fromPretrained("mouri45/parakeet-tdt_ctc-0.6b-ja-bf16")

Created as part of the PerchAgent project (Issue0023).