Ternary Bonsai 2 27B — MTP bundles (GGUF)
PrismML's Ternary Bonsai 2 27B (a 27B dense model with end-to-end ternary weights, Qwen3.8-27B hybrid backbone, 262K context, Apache-2.0) with a NextN / MTP draft head bundled in, so the model self-speculates — no second file, no external drafter.
The published PrismML GGUFs carry no NextN layer. These bundles add the upstream
Qwen/Qwen3.8-27B MTP head (its mtp.*
tensors, exported by upstream llama.cpp convert_hf_to_gguf.py --mtp --outtype q8_0)
as one extra layer; every other key and tensor is the PrismML file, byte for byte.
The head stays in the normal basis (it is not among the Hadamard-rotated tensors);
its token-embedding lookup on the rotated table gets the inverse transform in the
engine (PrismML's 518ad108).
| File | Weights | Size | sha256 |
|---|---|---|---|
Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf | PQ2_0 (ternary, ~1.72 bpw) + q8_0 head | 7.66 GB | 211653d24a0bfdc3d1c5f65711b995f9311ce6b24363cfed3b9b113087de18b3 |
Ternary-Bonsai-2-27B-PTQ1_0-MTP.gguf | PTQ1_0 (packed ternary) + q8_0 head | 6.40 GB | 174c9c6584e118766d5511e00a84fd8a31d146248915a726cfac8e6d140c6204 |
The vision projector is not duplicated here: use PrismML's
Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf (600 MiB) from the base repo with --mmproj.
Running
These files need an engine that runs ternary weights and a bundled NextN head
on a qwen35 hybrid: opencoti's llamafile
(from the c11 cut) does; PrismML's own llama.cpp fork runs the ternary weights but
not the bundled head.
# opencoti llamafile — NextN self-speculation; the draft depth for the ternary
# file types is a built-in table row (depth 2), no flag needed.
./opencoti-llamafile --server -m Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf \
-ngl 99 -fa on --spec-type draft-mtp
# with image input
./opencoti-llamafile --server -m Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf \
--mmproj Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf -ngl 99 -fa on --spec-type draft-mtp
The opencoti setup wizard lists this repo under the Ternary Bonsai 2 family and offers the projector as an optional download (it is loaded at the first image request, nothing is reserved for it at boot).
What MTP buys (greedy, 256-token answers, 12 chat prompts, medians)
| Card | PQ2_0, MTP off | PQ2_0, MTP on (depth 2) | acceptance |
|---|---|---|---|
| RTX PRO 6000 (Blackwell) | 128 tok/s | 161 tok/s | ~2.3 tokens / step |
| RTX 3090 | 46 tok/s | 57 tok/s | ~2.3 tokens / step |
Speculation preserves the model's distribution, not the argmax at near-ties: with greedy sampling an answer can differ from the unspeculated one at a position where the top-2 tokens are within a few hundredths of a nat (the verify batch rounds differently from single-token decode). No degeneration was measured (distinct 4-gram ratio identical on and off).
Reproducing the bundle
- Export the head:
convert_hf_to_gguf.py --mtp --outtype q8_0 Qwen/Qwen3.8-27B(upstream llama.cpp) →mtp-Qwen3.8-27B-Q8_0.gguf(c9e5c641de0c9896641c07a400ed92075024fe2c623549d427b1a6e44075bc0a). - Bundle it into the PrismML GGUF with opencoti's
bonsai2-mtp-bundle.py: the head becomes layerblock_count,nextn_predict_layers = 1; the script refuses unless every sharedqwen35.*hyperparameter and the vocab agree and no tensor collides. The result is byte-identical across machines (the two sha256 above).
License
Apache-2.0, as the base model and the Qwen3.8-27B head. Credits: PrismML for Bonsai 2, the Qwen team for Qwen3.8-27B.