GigaAM v3 RNNT — MLX (Apple Silicon)
MLX conversion of GigaAM v3 RNNT for on-device Russian speech recognition on Apple Silicon. Character-level RNNT model (33 Russian letters + blank): lowercase text without punctuation. Lower WER than CTC, sequential greedy decode.
- Source:
ai-sage/GigaAM-v3,rnntrevision - Architecture: 16-layer Conformer encoder (d_model 768, 16 heads, rotary positions)
- Input: mono 16 kHz audio
Usage
The runtime and converter live in mlx_convert/ of misteral/GigaAM
(proposed upstream in salute-developers/GigaAM#62).
git clone -b feat/mlx-clean https://github.com/misteral/GigaAM && cd GigaAM/mlx_convert
pip install mlx safetensors numpy huggingface_hub # ffmpeg is used by load_audio
hf download al-bo/gigaam-v3-rnnt-mlx --local-dir ./gigaam-v3-rnnt-mlx
from gigaam_mlx import load_model, load_audio # mlx_convert/gigaam_mlx.py
model = load_model("./gigaam-v3-rnnt-mlx")
print(model.transcribe(load_audio("audio.wav")))
Evaluation
Apple M4 Pro (24 GB), mlx 0.32.2, torch 2.14.0; 200 Golos Crowd test clips (823.85 s) — the sample of
garrrikkotua/gigaam-vs-whisper-russian, verified by SHA-256.
Method and full tables: mlx_convert/eval.
- Parity with upstream PyTorch (CPU, fp32): identical transcripts on 200/200 clips; encoder output max |Δ| 3.2e-06 median / 1.7e-05 max per clip.
- WER 1.50% (15 errors / 1001 words), CER 0.24%.
- Speed, × realtime (200 clips one by one, median 3.9 s / ≈20 s segments): MLX 91 / 102, PyTorch MPS 37 / 40, PyTorch CPU 10 / 39.
Files
model.safetensors— fp16 MLX weights (423 MB)config.json— architecture + vocabulary
The upstream checkpoint is stored in fp16, so the fp16 weights here lose nothing on the encoder.
Notes for other runtimes
If you run these weights with your own implementation, two details differ from a textbook Conformer:
- RoPE is applied to the raw input, split into heads, before the Q/K projections (value is not rotated).
- The RoPE base is 5000 (
pos_emb_max_len), not 10000: upstream callsRotaryPositionalEmbedding(d_model // n_heads, pos_emb_max_len)and the second positional parameter isbase. With 10000 transcripts still match on most clips, but encoder outputs are off by up to ~1.8.
Other details: log-mel uses mel_filterbank and stft_window stored in model.safetensors (taken from the PyTorch
model, center=False, log(clip(x, 1e-9, 1e9))); Conv1d weights are in MLX layout [out, K, in]; the LSTM bias is
bias_ih + bias_hh; blank = len(vocabulary).
License
Model weights: MIT, per the upstream ai-sage/GigaAM-v3 card. Conversion code: MIT. Original project: salute-developers/GigaAM.