Ternary Bonsai 2 27B — Abliterated (PTQ1_0 GGUF)
Refusal-ablated Ternary Bonsai 2 27B, edited directly on the ternary lattice and shipped in PrismML's native PTQ1_0 pack.
- 5,946,648,928 bytes — byte-for-byte the size of the official PTQ1_0 release
- No requantization, no BF16 bake, no runtime steering, no control vector at serve time
- Requires the PrismML llama.cpp fork
Also available: PQ2_0 (2.13 bpw), with optional MTP speculative decoding — same method, larger pack, and about 37% faster decode when you enable the grafted MTP head. Worth noting: the MTP head does not help this PTQ1_0 build (measured +1.6%), so the speedup is a reason to take the 2-bit pack, not this one.
Differs from PrismML's release in exactly 98 tensors and nowhere else. The other 753
tensors are byte-identical, including token_embd and output. Anyone with both files can
check that without trusting us.
Results
Paired against the untouched PrismML PTQ1_0 release: same runtime, same flags, greedy decoding, thinking off, 2,048-token answers. Judge is Qwen3.8-27B-OBLITERATED-Q8_0 with the StrongReject rubric. Numbers are from our 23 Sep 2026 comparison of every public Bonsai 2 abliteration, all run through identical scripts.
Refusal — SimpleSafetyTests, n=100
| stock | abliterated | |
|---|---|---|
| judged refusal | 83% | 0% |
| answer quality (StrongReject, 0–1) | 0.163 | 0.776 |
| empty completions | 0 | 0 |
Benign over-refusal — XSTest-safe, n=250 (lower is better)
| stock | abliterated | |
|---|---|---|
| over-refusal | 1.6% | 0.0% |
| partial refusal | 0.4% | 0.0% |
Over-refusal went down, not up. The model answers every harmless prompt in the set, which the stock model does not.
Capability — paired per question against stock PTQ1_0
| stock | abliterated | change | exact McNemar p | |
|---|---|---|---|---|
| MMLU, all 14,042 questions (0-shot) | 0.7802 | 0.7814 | +0.12 pp | 0.51 |
| HumanEval-164, greedy pass@1 | 0.890 (146/164) | 0.860 (141/164) | −3.0 pp | 0.18 |
HumanEval per problem: 139 both pass, 16 both fail, 2 gained, 7 lost. Neither change is statistically significant, but HumanEval points the wrong way, and at n=164 a real drop of a few points can't be ruled out.
Correction, 23 Sep 2026: this card previously reported HumanEval 0.811 → 0.805 and "no detectable capability change". That run gave each request only 512 tokens of context (
-c 4096split across 8 parallel slots), so many stock failures were answers cut off mid-code, not wrong answers. The table above is the rerun with that fixed.
The empty-completion and unparsed counts are listed deliberately. An empty answer scores as compliance under any prefix grader, so a 0% refusal rate means nothing without them. An earlier run of this same eval produced "0/100 refusal" purely from 82 empty answers, because thinking consumed the token budget before the model reached its reply.
Why this is not a normal abliteration
Standard abliteration projects a refusal direction out of the residual-writing tensors:
W ← W − λ·r·(rᵀW). On a ternary checkpoint that does nothing at all.
Bonsai 2's weights take three values per group of 128: {−s, 0, +s}. Projection asks each
weight to move about 1.4% of its magnitude while the lattice step is 100%, so repacking
rounds every weight straight back. Measured on real writer tensors:
| approach | refusal component removed | ternary digits changed |
|---|---|---|
| project, repack at λ=1.0 | none | 0 of 89,128,960 |
| project, repack at λ=1.3 | none | 72 of 89,128,960 (0.0001%) |
| refit per-group scales, digits frozen | 1.1% | 0 |
| flip digits on the lattice (this model) | 99.1–99.3% | 0.24% of edited tensors |
This is a trap worth naming. A projected-then-repacked model passes a file-size check and a quantizer reproducibility check while containing no abliteration whatsoever. If you build one of these, diff the tensors against the base release. Size is not evidence.
The per-group scales cannot carry the edit either — a scale can only scale, while the correction it must express varies across the 128 weights in its group. That is the 1.1% row.
Method
The quantity to remove is not each weight but the row sum c = rᵀW, one value per column. A
single flipped digit moves c by r_i · s · Δ, a whole lattice step — enormous next to the
1.4% nudge projection wanted. So the edit does not need many digits, it needs the right ones:
spend them where |r_i| is largest, since those buy the most change in c per unit of
damage to W. Greedy, one pass over rows in descending |r_i|, clipped so every digit stays
in {−1,0,+1} and every group scale is untouched.
- Writers only:
ffn_down,ssm_out,attn_output— 98 tensors - Blocks 15–63. Early blocks deliberately untouched; including them destroys coding.
Corroborated three ways: huihui's Qwen3.8 recipe skips 0–14, our own all-blocks arm
collapsed to 0.233 answer quality, and
Hikari07jprestored L0–7 on Bonsai 1 to recover coding from 16/20 to 19/20. - λ = 1.0, top 512 rows per tensor: 0.24% of the digits in the 98 edited tensors, which is 0.052% of all ternary digits in the model
- Direction transferred from the Qwen3.8-27B parent, recovered from huihui's released weights by rank-1 decomposition (σ₁/σ₂ ≈ 23)
Cost, stated plainly: |dW|/|W| = 0.071, roughly 5× what a true projection costs on a
full-precision model, because a coarse lattice cannot make small corrections. What that cost
bought: no measurable MMLU change, and a HumanEval drop too small to confirm at n=164.
Rebuilding this
Default llama-quantize does not reproduce the release — it yields 7.11 GB because
output.weight and token_embd.weight land at Q6_K/Q4_K. The released configuration is:
llama-quantize \
--output-tensor-type PTQ1_0 --token-embedding-type PTQ1_0 \
--tensor-type ssm_alpha=bf16 --tensor-type ssm_beta=bf16 \
in-F16.gguf out-PTQ1_0.gguf PTQ1_0 16
The 96 ssm_alpha/ssm_beta tensors cannot be reproduced from the released F16 at all: the
PTQ1_0 release stores them BF16 (7 mantissa bits) while the F16 file holds 10, so packing
truncates them ~0.4%. They are spliced verbatim from the base release here.
Usage
git clone https://github.com/PrismML-Eng/llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j
./build/bin/llama-server \
-m Ternary-Bonsai-2-27B-Abliterated-PTQ1_0.gguf \
-ngl 99 -c 8192 --flash-attn on --jinja
Benchmarks above were run with thinking off (--chat-template-kwargs '{"enable_thinking": false}').
Thinking mode has a known problem on this build. Leave thinking off for now. On safe
prompts it reasons about as long as stock. On harmful prompts it often drafts its answer
inside the reasoning and never finishes: with a 4,096-token budget and the default sampler
settings (temperature 1.0, top-k 20, top-p 0.95), 24% of harmful prompts got no answer at
all (stock: 0%), and 20% of safe prompts got no answer (stock: 10%). Greedy decoding makes
it worse: the reasoning falls into repeated sentences and 63% of harmful prompts get no
answer. A repetition penalty (--repeat-penalty 1.1 --repeat-last-n 2048) did not help at
default settings (25% no answer). A fix is being tested.
Correction, 23 Sep 2026: this section used to say the model "reasons ~25% longer than stock". That was wrong. Normal reasoning length is unchanged; the problem is runaway reasoning on harmful prompts.
Limitations
- Abliteration is a behavioural change, not a safety property. This model will produce unsafe content if asked, including in categories stock refuses outright. Use under your own policy and applicable law.
- Requires the PrismML fork; stock llama.cpp cannot load PTQ1_0.
- The direction is transferred from the Qwen3.8 parent, not fitted on Bonsai 2 itself. A natively fitted direction may do better and has not been tried.
- Same-condition lab measurement, not a leaderboard reproduction. Single greedy sample per problem; at n=164 the resolution floor is ~6.5 points.
- Vision mmproj not included (text only).
Prior art and attribution
- Base weights and format: prism-ml/Ternary-Bonsai-2-27B-gguf (Apache-2.0)
- Hikari07jp/Ternary-Bonsai-27B-Abliterated-LowDeg-GGUF did this first, on Bonsai 1, with the same family of method (direct ternary code edit on writers). Their early-block rollback is the degradation fix and is prior art for the block choice here. This release is Bonsai 2 and adds the measured mechanism: why projection fails on ternary weights, and what the lattice costs.
- Architecture lineage: Qwen3.5 / Qwen3.8-27B hybrid attention
License
Apache-2.0, same as the base release.