Granite Speech 5.0 470M TurboCTC MLX — Q8
This repository contains 8-bit affine weight quantization, group size 128 with group size 64 for encoder.input_linear for native Apple-silicon inference with the Granite-MLX Swift runtime.
It is converted from IBM's ibm-granite/granite-speech-5.0-470m-turboctc at revision 7e74c6438b7cfb5090cb6a131538f5e8515a7de3. The source model.safetensors SHA-256 is 8b98a8c34fd5fcb081caef719638eded31bb6d197d62053eefc5c1703aaf1ad4.
The conversion transposes PyTorch depthwise Conv1d kernels into MLX layout, removes training-only batch counters, converts retained floating-point tensors to FP16, and applies weight-only affine quantization where applicable. Activations remain floating point at runtime.
Family comparison
| Variant | Repository | Weight file | Size vs source | Agreement with source |
|---|---|---|---|---|
| IBM source | ibm-granite/granite-speech-5.0-470m-turboctc | 902.35 MiB | 100.00% | 100.0000% |
| FP16 | iky1e/granite-speech-5.0-470m-turboctc-mlx-fp16 | 902.22 MiB | 99.99% | 99.9706% |
| Q8 (this repository) | iky1e/granite-speech-5.0-470m-turboctc-mlx-q8 | 466.03 MiB | 51.65% | 99.8825% |
| Q6 | iky1e/granite-speech-5.0-470m-turboctc-mlx-q6 | 367.51 MiB | 40.73% | 99.6989% |
| Q5 | iky1e/granite-speech-5.0-470m-turboctc-mlx-q5 | 311.22 MiB | 34.49% | 99.4712% |
| Q4 | iky1e/granite-speech-5.0-470m-turboctc-mlx-q4 | 254.93 MiB | 28.25% | 98.4135% |
“Agreement with source” is word-level transcript agreement, calculated as 100 − Levenshtein word edits / source words. It is not WER and does not measure correctness against a human transcript. The reference is the matching original IBM checkpoint loaded by the same native Swift runtime.
The test recording is a 6,118.72-second (101m58.72s) single-speaker Stanford CME295 lecture. All checkpoints used Granite-MLX's bounded-memory defaults: 122.88-second chunks, 20.48-second context, FP16 activations, greedy CTC decoding, and a 64 MiB MLX cache. The source transcript contained 13,615 words. Raw benchmark JSON and transcripts are preserved with the Granite-MLX project and will be published with its source repository.
Usage
granite-mlx /path/to/audio-or-video --model iky1e/granite-speech-5.0-470m-turboctc-mlx-q8
Granite-MLX accepts common audio and video files, downloading this repository automatically on first use. It can export TXT, SRT, WebVTT, JSON, or all formats.
Reproducing the conversion
uv run python Scripts/convert_granite.py \
/path/to/source-checkpoint \
/path/to/output-q8 \
--precision fp16 \
--quantization-bits 8 \
--group-size 128 \
--source-model-id ibm-granite/granite-speech-5.0-470m-turboctc \
--source-revision 7e74c6438b7cfb5090cb6a131538f5e8515a7de3
The publication converter (Scripts/convert_granite.py) has SHA-256 83141299b6e680fbdc020aed6674683b5cac03685dec839a30767f55de0942fb. Validation used Granite-MLX's native Swift release build with Apple Swift 6.2.3, MLX Swift 0.31.4 (dc43e62d7055353c7f99fa071a4e71d29dfddc44), swift-transformers 1.3.3 (2fa33e1f5e7131a7fc64c28e6d161dcec0d24820), macOS 26.5.2, and Xcode 26.2.
License
These converted model weights retain the source model's Apache 2.0 terms. The Granite-MLX Swift software is separate from the weights; this model repository does not set the software's license.