Ternary-Bonsai-27B-antidoom-DSpark
A DSpark-accelerated + antidoom
(FTPO anti-repetition) build of prism-ml/Ternary-Bonsai-27B —
the ternary (Q2_0, 2.125 bpw g128) quantization of the Qwen3.5-27B hybrid-attention model.
Runs in the PrismML-Eng/llama.cpp fork (CUDA Q2_0 hybrid-attention kernels). Built and validated on a single RTX 5090 (32 GB, Blackwell sm_120).
Companion model: Danny-Dasilva/Bonsai-27B-antidoom-1bit-DSpark (1-bit).
Contents
| File | What |
|---|---|
Ternary-Bonsai-27B-antidoom-Q2_0.gguf | antidoom-tuned ternary model (~7 GB) |
Ternary-Bonsai-27B-dspark-Q4_1.gguf | DSpark speculative drafter (from prism-ml) |
Ternary-Bonsai-27B-mmproj-Q8_0.gguf | vision projector (from prism-ml) |
Run with DSpark (fastest)
build/bin/llama-server \
-m Ternary-Bonsai-27B-antidoom-Q2_0.gguf \
-md Ternary-Bonsai-27B-dspark-Q4_1.gguf \
--spec-type draft-dspark --spec-draft-n-max 4 \
-ngl 999 -ngld 999 -fa on -c 16384 -np 1
Speed comparison — all four models (RTX 5090, 32 GB)
Single stream, greedy, 256 new tokens, PrismML llama.cpp fork (CUDA 13 / sm_120).
Native = llama-bench tg256 (fa=1, r=2). DSpark = llama-speculative-simple,
--spec-draft-n-max 4, code prompt (high-acceptance workload); all four runs measured
back-to-back in the same session.
| Model | Native tok/s | + DSpark tok/s | Accept | DSpark speedup |
|---|---|---|---|---|
| Bonsai-27B 1-bit (base) | 166.0 | 208.3 | 78.6% | 1.25× |
| Bonsai-27B 1-bit antidoom (this family) | 153.7 | 192.4 | 74.6% | 1.25× |
| Ternary-Bonsai-27B (base) | 136.5 | 185.9 | 69.2% | 1.36× |
| Ternary-Bonsai-27B antidoom (this family) | 129.2 | 184.6 | 69.2% | 1.43× |
tok/s (code prompt, bs=1) native ▒ +DSpark █
Bonsai 1-bit base ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 166.0
████████████████████████████████ 208.3
Bonsai 1-bit antidoom ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 153.7
█████████████████████████████ 192.4
Ternary base ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 136.5
████████████████████████████ 185.9
Ternary antidoom ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 129.2
████████████████████████████ 184.6
Notes:
- DSpark's gain is workload-dependent: on low-acceptance prose (~51%) the 1-bit target can be slower than native because the tiny weights make drafting overhead relatively expensive.
- The antidoom variants are ~5–8% slower natively because re-quantization stored the output tensor at q6_K (prism's originals pack it smaller); with DSpark the gap mostly closes.
- FTPO does not hurt the DSpark drafter (accept 74.6–78.6% 1-bit, 69.2% ternary, unchanged base vs antidoom for ternary).
- All numbers beat prism-ml's published H100 figures (104.8→143.8 1-bit, 98.0→131.8 ternary): these models are memory-bandwidth-bound and the 5090 has more bandwidth headroom at bs=1.
Antidoom (FTPO) pass — what was done
Bonsai models are unusually doom-loop resistant (they self-correct repetition), so runaway-repetition pairs were surfaced by priming: contexts that already began repeating a phrase, sent to the raw completion endpoint, where the model continues the loop. Antidoom's own detector + chosen-token sampler extracted 40 FTPO preference pairs (rejected = the loop-continuation token, chosen = coherent escapes). FTPO trained a LoRA (r=64, q/k/v/o + gate/up/down + lm_head, 12 epochs, lr 3e-5) on the FP16 unpacked model:
- chosen-win (prefers the coherent escape over continuing the loop): 0.02 → 0.53 (early-stopped ≥0.4).
- LoRA merged (CPU) into FP16, re-quantized to
Q2_0. The aggressive 2-bit format partially dilutes the LoRA's fine adjustments; the anti-repetition effect is strongest at higher precision.
Provenance
Base weights, DSpark drafter and mmproj are from prism-ml (model weights re-quantized after the antidoom LoRA merge; drafter/mmproj copied unchanged). Built with antidoom and the PrismML llama.cpp fork.