Bonsai-27b-Ternary-CRACK-GGUF

Native Q2_0 GGUF release of the Bonsai 27B CRACK model for the Prism llama.cpp fork. This repo contains the quantized language model and the matching F16 Qwen3VL multimodal projector.

Files

The projector contains 334 tensors and both temporal patch slices v.patch_embd.weight and v.patch_embd.weight.1 (temporal_patch_size=2). The original image and video processor configuration files are included. Image input is live-tested. Direct video-container input depends on the Prism runtime surface; extract frames or use a compatible Qwen3VL video client when the CLI does not accept the container directly. See preprocessor_config.json and video_preprocessor_config.json for the retained preprocessing metadata.

Runtime

These native low-bit types require the Prism fork:

git clone https://github.com/PrismML-Eng/llama.cpp
cd llama.cpp
cmake -B build && cmake --build build -j

Text:

./build/bin/llama-cli \
  -m Bonsai-27b-Ternary-CRACK-Q2_0.gguf \
  --reasoning off -ngl 99 -n 256 \
  -p "Explain quantum computing in simple terms."

Image/VL:

./build/bin/llama-cli \
  -m Bonsai-27b-Ternary-CRACK-Q2_0.gguf \
  --mmproj mmproj-Bonsai-27b-Ternary-CRACK-F16.gguf \
  --image image.jpg --reasoning off -ngl 99 -n 256 \
  -p "Describe the image precisely."

Server:

./build/bin/llama-server \
  -m Bonsai-27b-Ternary-CRACK-Q2_0.gguf \
  --mmproj mmproj-Bonsai-27b-Ternary-CRACK-F16.gguf \
  --reasoning off --host 0.0.0.0 --port 8080 -ngl 99

Verified evaluation

MMLU is the same fixed, stratified 200-question sample for all rows. It uses next-token A/B/C/D logits with reasoning/thinking disabled; it is not a claim for the complete official MMLU suite. HB is the complete unsliced 320-case text run. The live text smoke measured 43.74–44.10 tok/s and the image request measured 44.08 tok/s on an Apple M5 Max using the Prism Metal runtime.

Architecture and compatibility

• 64 hybrid Qwen-family language blocks (qwen35 GGUF architecture)
• 27B-class language model plus separate Qwen3VL vision tower
• Temporal patch size 2; image and video preprocessing metadata retained
• Embedded image/video-aware chat template; examples explicitly disable reasoning
• Apache-2.0; see LICENSE.txt and NOTICE.txt

한국어 안내

이 저장소는 Bonsai 27B CRACK 모델의 네이티브 Q2_0 GGUF 배포본입니다. Prism llama.cpp 포크가 필요하며, 텍스트 모델과 F16 Qwen3VL 멀티모달 프로젝터를 함께 제공합니다. 위 MMLU 점수는 추론을 끈 200문항 로짓 평가이고, HB 점수는 전체 320문항 결과입니다. 이미지 입력은 실제로 검증했으며 비디오 컨테이너 입력은 사용 중인 Prism 클라이언트의 지원 여부를 확인해야 합니다.

Lineage

• Prism GGUF base: prism-ml/Ternary-Bonsai-27B-gguf
• Upstream family: Qwen/Qwen3.6-27B
• Publisher: dealignai