GigaChat Audio MLX — BF16
English · Документация на русском
Native BF16 MLX conversion of
ai-sage/GigaChat3.1-Audio-10B-A1.8B
for Apple Silicon. It accepts audio plus a text instruction and produces text:
audio QA, transcription, translation, emotion analysis, temporal grounding,
and timestamped summaries.
This is an independent port, not an official release by the upstream authors.
GitHub · PyPI · All models · Full results · Upstream model · Paper
Models
gigachat-audio-mlx— BF16 · best measured quality ← this modelgigachat-audio-mlx-q8-bf16— Q8/BF16 · smaller and faster
| Model | Choose it for | Model size | Measured process memory | Processing 5 min 17 sec* |
|---|---|---|---|---|
| BF16 · this model | Best measured answer quality | 22.54 GB | 22.84 GiB | 10.41 s |
| Q8/BF16 | Less disk and memory, higher speed | 13.75 GB | 14.80 GiB | 6.90 s |
Choose Q8/BF16 when size, memory, and speed matter; choose BF16 for the best
measured answer quality. Both variants use the same
gigachat-audio-mlx runtime.
The original model and paper are by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin. The upstream model card declares MIT. See the runtime license and third-party notices.
What do I need to run it?
| Model | Model size | Free disk to set aside | Measured whole-process memory | Processing 5 min 17 sec* |
|---|---|---|---|---|
| BF16 | 22.54 GB | about 25 GB | 22.84 GiB | 10.41 s · 30.5× realtime |
The free-disk figure is a rounded practical reserve; the CLI dry run reports the exact value. Memory is measured use, not a minimum-memory guarantee. This release is verified only on a 48 GB Mac; 16/24/32 GB support is not yet claimed.
Why MLX matters on this Mac
Same Apple M4 Pro / 48 GB. Original is a stopped memory-safety smoke, not a completed speed benchmark; MLX values are medians of five clean runs.
| Runtime | Audio | Outcome | Whole-process footprint |
|---|---|---|---|
| Original PyTorch BF16 on MPS | 9.84 s | No answer after 277 s; stopped | at least 51.92 GiB |
| MLX BF16 | 317.27 s | Completed in 10.41 s | 22.84 GiB |
| MLX Q8/BF16 | 317.27 s | Completed in 6.90 s | 14.80 GiB |
The pinned upstream Transformers path is demonstrated for CUDA. On MPS it exceeded this Mac's safe memory budget and nearly filled system swap during the short smoke. The 5-minute Original run was not started, so no speedup ratio is claimed. MLX is what makes the model usable on the tested Mac.
Quick start
brew install uv
uv tool install gigachat-audio-mlx
gigachat-audio-mlx doctor
gigachat-audio-mlx download --variant bf16 --dry-run
gigachat-audio-mlx download --variant bf16 --yes
The dry run reports the exact download size, cache hits, free disk space, and destination before any network transfer. The runtime never executes remote model code.
gigachat-audio-mlx run meeting.m4a \
--variant bf16 \
--task summarize \
--max-tokens 768
Exact task examples
The text instruction is part of every input. It must not be omitted when interpreting an answer.
Russian speech transcription
Prompt:
Transcribe the Russian speech exactly. Return only the transcript.
Output:
Всё во Вселенной состоит из материи. Вся материя состоит из мельчайших частиц, называемых атомами.
Temporal grounding
Prompt:
Укажи таймкод начала и конца речи в аудио.
Output:
Таймкоды начала и конца речи в аудио:
- Начало: 00:00
- Конец: 00:20
Artifact
| Field | Value |
|---|---|
| Format | sharded safetensors, data-only |
| Runtime payload | 22,539,048,633 bytes |
| Precision | BF16 |
| Source revision | bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb |
| Runtime | gigachat-audio-mlx==0.1.4 |
| Verified target | Apple M4 Pro / 48 GB / macOS 15.7.7 |
| Weight files commit | 7484ac64355654f899fefa4e45637dc53a0ba40c |
| Manifest SHA-256 | 3a845299123f35ba65313fb19e5f030516dc84d53b791541c7657dac91a0f8a3 |
The model repository contains no executable Python. model_manifest.json
pins the source lineage and SHA-256 of every data file. Loading fails on a
missing, changed, or unexpected artifact.
Results available today
The BF16 port is the accepted native MLX baseline. In the frozen 50-item parity gallery, its decoded text exactly matched the original PyTorch BF16 output on 41/50 inputs. This is an implementation-parity diagnostic, not an aggregate quality score.
What changes with Q8/BF16
Lower is better in every row. Both models used the same MLX workloads.
| Metric | MLX BF16 | MLX Q8/BF16 | Change |
|---|---|---|---|
| Model size, decimal GB | 22.54 | 13.75 | −39.0% |
| Whole-process memory, 5-minute profile | 22.84 GiB | 14.80 GiB | −35.2% |
| End-to-end time, 5-minute profile | 10.41 s | 6.90 s | −33.7% |
| Responses with a factual error, lower is better | 26/64 (40.63%) | 29/64 (45.31%) | +3 responses (+4.69 pp) |
The 5-minute workload contains 317.27 seconds of audio. Resource values are the median of five clean processes on the verified M4 Pro / 48 GB machine.
Q8/BF16 is the smaller, faster option; BF16 provides the best measured answer quality. In the blinded review, Q8/BF16 had factual errors in 3 more cases.
From Original to MLX
| Check | Original PyTorch BF16 | MLX BF16 | MLX Q8/BF16 |
|---|---|---|---|
| Model size, decimal GB | 23.73 | 22.54 | 13.75 |
| Exact decoded-text agreement | Reference | 41/50 vs Original | 45/50 vs MLX BF16 |
The post-publication Original PyTorch/MPS resource check did not complete even
the 9.84-second smoke on this 48 GB Mac. After 277 seconds its process footprint
had reached at least 51.92 GiB and system swap was nearly full, so it was stopped
before the 5-minute run. Original therefore remains N/A in the direct Mac RAM
and speed table; no latency is inferred from a failed, swap-dominated run.
Results reported by the Original authors
These values come from the upstream model card; we did not rerun them.
| Task and dataset | Metric | Original reported result |
|---|---|---|
| Audio QA · MMAU | Accuracy ↑ | 62.2 |
| Audio QA (Russian) · RuBQ | Accuracy ↑ | 60.0 |
| Emotion · Dusha podcast | Accuracy ↑ | 92.4 |
| ASR (Russian) · FLEURS ru | WER ↓ | 4.4 |
| Translation · FLEURS ru→en | BLEU ↑ | 33.4 |
| Temporal localization · 20–60 min | mIoU ↑ | 48.3 |
| Timed summary · 20–60 min | Overall ↑ | 67.9 |
Full public release evidence separates ASR, translation, emotion, temporal localization, timed descriptions/summaries, factual review, memory, and speed. It does not collapse these tasks into one number. See the full results table.
Limitations
- Verified only on Apple M4 Pro with 48 GB unified memory and macOS 15.7.7.
- No claim is made for 16/24/32 GB machines or every M1–M5 generation.
- No 90-minute-audio support claim is made in this release.
- Generated facts, emotion labels, summaries, and timestamps can be wrong and must be checked against the source audio.
- Long summaries may require a larger output-token budget, which also changes the memory estimate.
- No diarization, streaming microphone mode, speech synthesis, training, or fine-tuning is included.
- Do not use the output as the sole basis for medical, legal, employment, safety, or surveillance decisions.
Users are responsible for consent, privacy, copyright, and dataset terms for audio they process.
License and attribution
- Upstream model: https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B
- Upstream revision:
bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb - Paper: GigaChat Audio: Time-aware Large Audio Language Model
- Original authors: Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
- Upstream license declaration: MIT in the upstream model card
- Runtime and MIT license: https://github.com/ai-babai/gigachat-audio-mlx
- Model collection: https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437
Please cite both the original paper above and this project through
CITATION.cff.
See the upstream model card and
THIRD_PARTY_NOTICES.md
for full attribution and provenance.