Ternary-Bonsai-1.7B-mlx-2bit
Ternary (1.58-bit) language model for Apple Silicon
7.2x smaller than FP16 | 3.8x faster on M4 Pro | 103 tok/s on iPhone | runs on Mac, iPhone, iPad
Highlights
- 0.45 GiB (0.48 GB) packed 2-bit size (down from 3.44 GB FP16) — fits anywhere
- Ternary weights {-1, 0, +1} across embeddings, attention projections, MLP projections, and LM head
- 57.5 avg benchmark score across 6 categories
- 103 tok/s on iPhone 17 Pro Max
- MLX-native format with group size 128 and FP16 scaling
Resources
- White Paper
- Demo repo — examples for serving, benchmarking, and integrating Bonsai
- Discord — community support and updates
- Kernels: MLX (Apple Silicon) · mlx-swift (iOS/macOS) — 2-bit format is supported out of the box
Model Overview
| Item | Specification |
|---|---|
| Base model | Qwen3-1.7B |
| Parameters | 1.72B |
| Architecture | GQA, SwiGLU MLP, RoPE, RMSNorm |
| Context length | 32,768 tokens |
| Vocab size | 151,936 |
| Weight format | Ternary g128: {-1, 0, +1} with FP16 group-wise scaling |
| Packed 2-bit size | 0.45 GiB (0.48 GB) |
| Ternary coverage | Embeddings, attention projections, MLP projections, LM head |
| License | Apache 2.0 |
Quantization Format: Ternary g128
Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights:
w_i = scale_g * t_i, t_i in {-1, 0, +1}
The information-theoretic cost is log2(3) ≈ 1.585 bits per weight, plus FP16 group scales (16 bits per 128 weights), for a theoretical minimum of ~1.71 bits/weight. This release uses the MLX 2-bit format, which stores each ternary value in 2 bits plus group scales, for an effective ~2.125 bits/weight.
Memory
| Format | Size | Reduction | Ratio |
|---|---|---|---|
| FP16 | 3.44 GB | -- | 1.0x |
| MLX 2-bit g128 | 0.45 GiB (0.48 GB) | 86.0% | 7.2x |
Quickstart
MLX (Python)
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("prism-ml/Ternary-Bonsai-1.7B-mlx-2bit")
response = generate(
model,
tokenizer,
prompt="Explain quantum computing in simple terms.",
max_tokens=256,
)
print(response)
Throughput (MLX / Apple Silicon)
| Platform | Backend | PP512 (tok/s) | TG128 (tok/s) | FP16 TG (tok/s) | Speedup |
|---|---|---|---|---|---|
| M4 Pro 48 GB | MLX (Python) | 1,764 | 235 | 62 | 3.8x |
iPhone 17 Pro Max (MLX Swift)
| Platform | Backend | PP512 (tok/s) | TG128 (tok/s) | 4-bit TG (tok/s) | Speedup |
|---|---|---|---|---|---|
| iPhone 17 Pro Max | MLX Swift | 1,456 | 103 | 60 | 1.7x |
Benchmarks
Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100. Full benchmark suite (10 benchmarks):
| Model | Size | Avg | MMLU-R | MuSR | IFEval | GSM8K | HE+ | BFCLv3 |
|---|---|---|---|---|---|---|---|---|
| Ternary Bonsai 1.7B | 0.37 GB | 58.47 | 52.9 | 50.8 | 70.1 | 74.2 | 51.8 | 51.0 |
| 1-bit Bonsai 1.7B (prior) | 0.24 GB | 49.60 | 43.2 | 45.1 | 63.0 | 66.3 | 45.1 | 34.9 |
| Qwen3 1.7B | 3.44 GB | 66.57 | 66.8 | 50.1 | 70.3 | 83.1 | 57.3 | 71.8 |
| Qwen3 0.6B | 1.19 GB | 48.02 | 47.5 | 41.5 | 62.8 | 64.1 | 30.5 | 41.7 |
| LFM2 1.2B | 2.34 GB | 46.73 | 52.9 | 25.4 | 77.5 | 62.2 | 36.0 | 26.4 |
| Gemma3 1B | 2.00 GB | 45.53 | 43.2 | 37.0 | 61.9 | 64.4 | 40.2 | 26.5 |
| Llama 3.2 1B | 2.47 GB | 39.88 | 47.2 | 29.2 | 47.7 | 49.0 | 35.4 | 30.8 |
Intelligence Density
density = -ln(1 - score/100) / size_GB
| Model | Size | Intelligence Density (1/GB) |
|---|---|---|
| Ternary Bonsai 1.7B | 0.37 GB | 2.389 |
| 1-bit Bonsai 1.7B (prior) | 0.24 GB | 2.832 |
| Qwen3 0.6B | 1.19 GB | 0.549 |
| Qwen3 1.7B | 3.44 GB | 0.318 |
| Gemma3 1B | 2.00 GB | 0.304 |
| LFM2 1.2B | 2.34 GB | 0.269 |
| Llama 3.2 1B | 2.47 GB | 0.206 |
Citation
@techreport{ternarybonsai,
title = {Ternary Bonsai: 1.58-bit Language Models at 8B, 4B, and 1.7B Scale},
author = {Prism ML},
year = {2026},
month = {April},
url = {https://prismml.com}
}
Contact
For questions, feedback, or collaboration inquiries: contact@prismml.com