Ternary Bonsai 2 27B — MLX, lossless, 1.75 bits/weight, with vision

A lossless repack of Prism ML's Bonsai 2 27B for Apple Silicon. All 402 ternary language matrices stay packed at 1.75 bits/weight during inference, including their scale storage. Custom Metal kernels read the packed representation directly; the loader retains no expanded model-weight copy.

The original FP16 vision tower, processor, tokenizer, chat template, recurrent projections and rotation signs are preserved. This is the multimodal Bonsai 2 counterpart to our original Bonsai 27B release.

6.92 GB of weight payload; 1.68 GB less loaded MLX active memory than the official 2-bit checkpoint. This release is for memory savings and verified fidelity. The measurements below do not establish a speed improvement; prefill was slower on the primary workloads.

Run it

Tested on an M4 Max with 48 GB unified memory, using Python 3.12. Install the four pinned runtime packages and use the bundled loader. Stock mlx_lm.load, mlx_vlm.load, and generic Transformers loaders do not understand this custom artifact.

With uv:

uvx --from huggingface-hub==1.26.0 hf download \
  inductiveML/Ternary-Bonsai-2-27B-mlx-lossless-1.75bpw \
  --local-dir bonsai2
cd bonsai2
uv venv --python 3.12
uv pip install -r requirements.txt

# Text
.venv/bin/python generate.py --prompt "Explain what a ternary weight is." --max-tokens 128

# Image: the included OCR fixture reads ORCHID 4729
.venv/bin/python generate.py --prompt "Read the text in this image." \
  --image examples/ocr.png --max-tokens 32

# Multiple images
.venv/bin/python generate.py --prompt "Describe each image in order." \
  --image examples/spatial.png --image examples/second.png --max-tokens 128

Use --thinking to enable the source thinking template. The script defaults to greedy decoding (--temperature 0.0) with thinking disabled. Thinking consumes the same --max-tokens budget as the answer. Image paths are local files. For repeatable downloads, add --revision <commit> using the commit shown in this repository's history.

Python API

Run from the downloaded model directory, or add that directory to sys.path:

from pathlib import Path
from mlx_vlm import generate
from mlx_vlm.prompt_utils import apply_chat_template
from ternel_bonsai2_runtime import chat_config, load_vl_model

model_dir = Path(".")
model, processor, config = load_vl_model(model_dir)
images = ["examples/ocr.png"]  # use [] for text only
prompt = apply_chat_template(
    processor, chat_config(config), "Read the text in this image.",
    num_images=len(images), enable_thinking=False,
)
result = generate(
    model, processor, prompt=prompt, image=images or None,
    max_tokens=128, temperature=0.0, verbose=False,
)
print(result.text)

The self-contained ternel_bonsai2_runtime.py includes the Metal kernels; no Ternel installation or Prism runtime fork is needed. Runtime versions are MLX 0.32.0, MLX-LM 0.31.3, MLX-VLM 0.6.3 and Transformers 5.5.0. Other versions and other Apple Silicon hardware have not been validated for this release. This package provides local inference, not an OpenAI-compatible server.

What is lossless here?

This is a re-encoding of the released ternary checkpoint, with no retraining or additional quantization. Each group of 128 weights uses 26 bytes of base-3 codes plus 2 raw FP16 scale bytes: 28 × 8 / 128 = 1.75 bits/weight. The three values are {-scale, 0, +scale}. This is a ternary kernel, not a binary 1-bit kernel.

The 1.75-bit figure applies to the ternary matrices. The full multimodal payload also includes the unchanged vision and other floating-point tensors. Memory needed for activations, KV/recurrent state, Metal workspace and the allocator is additional.

GB means decimal GB. The ternary payload is 5,877,760,000 bytes and the unchanged vision payload is 921,460,192 bytes. Safetensors headers and repository assets add to download size. Loaded MLX active memory is not whole-process RSS or peak request memory.

The runtime preserves the source's 1024-wide Hadamard transforms and inverse embedding transform, FP32 intermediate arithmetic, FP32 projection activations, and FP16 embedding/vision behavior. It loads Prism's existing tensor ordering and normalization values directly. A forced FP16 projection-activation experiment failed the frozen vision drift limit and was rejected. It is not used here.

Validation

The complete conversion was independently reconstructed from the written shards:

• 26,869,760,000 / 26,869,760,000 ternary codes match across all 402 modules.
• All 209,920,000 raw FP16 scale groups match bit for bit.
• All 1,184 passthrough tensors are byte-identical, including vision and rotation signs.
• Malformed codes, missing signs, incorrect manifests, signed zero and subnormal scales have regression coverage.

The pinned Prism loader and packed loader were compared over 21 cases: the existing text suite, thinking, instruction and tool formatting, tokenwise cached generation, 17-token chunked prefill, 8,192-token context, OCR, spatial reasoning and multiple images.

Short prompts were checked at every position; longer prompts used 128 fixed, evenly spaced positions. The skip requires an unavailable original Bonsai 1 artifact snapshot. A fresh standalone check, with both Ternel package imports blocked, also read ORCHID 4729 correctly. The bundled CLI is checked with text and multiple images before publication.

Lossless weights do not promise bit-identical floating-point logits for arbitrary inputs. Different accumulation orders produce small numerical differences. This release has not rerun the upstream model's full capability benchmarks or validated its entire advertised context window. The reported agreement applies to the cases above.

See correctness.json, weight_identity.json, the conversion manifest, and release_checks.json.

Speed and peak memory

These are diagnostic measurements during normal desktop use on the M4 Max, not idle-machine performance claims. Each workload used isolated processes, two warmups, six alternating AB/BA rounds and five measured samples per arm per round, with 64 greedy output tokens. Confidence intervals resample whole paired rounds. All compared inputs and 64-token outputs matched.

Speedup is official-runtime time divided by packed-runtime time; below 1 means slower.

Primary text geometric means: 0.839× prefill, 1.011× decode. The unchanged configuration was also checked on held-out 256- and 1,024-token prompts: 1.043× prefill, 1.124× decode, with confidence bounds that still do not establish improvement on every workload. Primary image end-to-end latency was 1.011× (95% CI 0.934–1.242), so image no-regression is not established.

The original combined memory-and-speed gate remains NOT QUALIFIED in qualification.json. This publication accepts the demonstrated memory savings and correctness; it does not change that historical speed decision. There is no speculative decoding, KV-cache quantization or vision quantization in this comparison. Details: primary results, held-out results, frozen configuration.

Related implementations

Public repositories were checked on 2026-09-19. Bonsai 2 MLX already exists: Prism publishes the official 2-bit model, and JANG publishes a 1.75-bit download. JANG's inspected runtime expands that representation into 2-bit affine MLX arrays when loading, and its model card describes 6-bit vision linears. The monkeyinference project also contains native five-trit Metal experiments; the inspected implementation is text-only, defaults to 2-bit, and its optional five-trit path retains the 2-bit weights for larger batches.

We did not find a public release in the checked results combining native resident 1.75-bit language weights, no retained 2-bit model copy, and the unchanged FP16 vision tower. This is a scoped search result, not a claim to have invented ternary packing or proof that no equivalent exists. Pinned research sources.

Reproduce and audit

The source is prism-ml/Ternary-Bonsai-2-27B-mlx-2bit at revision 3f926b415992eaa2ae9dd7b573706494d6bbf787. The source safetensors SHA256 is 130de5925082c168b7866b2e91b52e44abbafc99017e3ca352b77b5b55a269ed.

The exact converter, runtime sources, evaluation code, tests and dependency lock are bundled in ternel-bonsai2-source.tar.gz. REPRODUCE.md explains conversion, configuration replay and correctness checks. SHA256SUMS covers the release files; the tensor manifest includes shard, tensor and source metadata hashes. The runtime is frozen at SHA256 9e733d926539e15091cca9b143b82318322d8680d9d5ee141043c8d8688764cf.

License and attribution

Apache 2.0; the source LICENSE and NOTICE.txt are included unchanged. Created using Bonsai by Prism ML. Bonsai 2 is built from Qwen3.8-27B by Alibaba Cloud; its MLX architecture implementation uses the qwen3_5 model family. Ternel supplies the lossless storage conversion and native packed execution. See the source model card for upstream model capabilities and limitations.