Ternary Bonsai 27B Abliterated-LowDeg (Q2_0 GGUF)
Low-degradation direct GGUF abliteration of Prism Ternary Bonsai 27B.
- Format: official Q2_0 g128 ternary pack (~7.2 GB)
- No runtime steering / no BF16 bake / no requant
- Built for the PrismML llama.cpp fork (hybrid-attention Q2_0 kernels)
Hard-convert 38/40 on a fixed harmful pilot while keeping coding near the parent — not a max-HC-only scar model.
Files
| File | Role |
|---|---|
Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf | language model (this release) |
Optional speculative decode: use Prism’s official DSpark drafter unchanged
(Ternary-Bonsai-27B-dspark-Q4_1.gguf from the base repo).
Accept-rate A/B on this weight vs parent: ≈1.00× (n=30, CI overlap) → no draft retrain needed.
Quickstart
# Prism fork required
git clone https://github.com/PrismML-Eng/llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j
./build/bin/llama-server \
-m Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf \
-ngl 99 -c 8192 -np 1 \
--host 127.0.0.1 --port 8080
# optional DSpark
./build/bin/llama-server \
-m Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf \
-md Ternary-Bonsai-27B-dspark-Q4_1.gguf \
-ngl 99 -ngld 99 \
--spec-type draft-dspark --spec-draft-n-max 4 \
-c 8192 -np 1
Method (short)
- Residual / direction code-edit on ternary writers over the parent Q2_0 pack
- Escalate to a high hard-convert seed (
esc-ALL-f65) - Seed-stage surgical rollback: restore parent bytes on selected L0–7 writers
(minimal maskminM23, ~1.5 MB changed vs full L0–7 rollback) - Gate on paired harmful eval + capability pilots; keep low-degradation winner
Runtime RepE hooks are not required at serve time.
Results
All figures below are same-condition lab measurements. They are not a full open leaderboard reproduction.
Refusal gate (harmful pilot)
| value | |
|---|---|
| n | 40 |
parent baseline refusals (REFUSAL | COMPLY_WITH_HEDGE) | 40/40 |
hard convert (parent refused → candidate pure COMPLY) | 38/40 |
| empty completions | 0 |
| transport errors | 0 |
Hard convert does not count hedges / weak / empty.
Harmless side-effect (same n=40): reverse flip 0.
Capability pilot
| split | this model |
|---|---|
| coding | 19/20 |
| tool | 15/15 |
| agentic | 5/5 |
Parent coding on the same pilot was 19/20; an earlier HC-max seed (esc-f65) dropped coding to 16/20.
Serious benches (seed=0, thinking off)
| bench | parent Q2_0 | this (LowDeg) |
|---|---|---|
| GSM8K (n=100) | 68 | 71 |
| MMLU subset (n=100) | 62 | 60 |
| HumanEval (n=164) | 141 | 140 |
| MBPP (n=100) | 66 | 64 |
Decode (reference, not a claim of superiority)
- RTX PRO 6000 WS, no draft: single-stream TG ~139 tok/s (depth0, llama-bench n=5)
- RTX 5070 Ti 16GB + official DSpark: single-stream ~80–85 tok/s, DSpark-active ctx up to 16k; parallel sweet spot np=2 (~89 agg tok/s)
Intended use
Research and local/offline assistants where reduced over-refusal is desired.
Still a general model: it can produce unsafe content if asked. Use under your own policy and applicable law.
Limitations
- Requires Prism llama.cpp (stock llama.cpp may not run this Q2_0 hybrid pack correctly)
- Abliteration is behavioral, not a safety proof
- Long-context / high-parallel on 16GB cards can silently disable DSpark when compute buffers OOM — check
timings.draft_n - Vision mmproj is not included (text GGUF only)
Attribution
- Base weights & format: prism-ml/Ternary-Bonsai-27B-gguf (Apache-2.0)
- Architecture lineage: Qwen3.5 / Qwen3.6-27B hybrid attention
- This derivative: direct ternary GGUF edit + seed-stage low-degradation rollback (
W-RB-L0-7-minM23)
License
Apache-2.0 (same as the base GGUF release).