TernaryQuench Qwen3.8-27B — GGUF
TernaryQuench gives you the complete recipe for building your own ternary language models, from generating calibration data to exporting the file you run.
This release compresses Qwen3.8-27B for local inference. The calibration tools, resumable trainer, exporters, and evaluation code are open source, so you can inspect the model's recipe and build your own.
Get started
With Ollama:
ollama run hf.co/penkia/TernaryQuench-Qwen3.8-27B-GGUF:Q2_K
Or with llama.cpp:
llama-cli -hf penkia/TernaryQuench-Qwen3.8-27B-GGUF:Q2_K -cnv -c 32768
The download is 10.86 GB. Allow additional memory for the context cache and runtime. Tested with Ollama 0.33.3.
For native MLX inference on Apple silicon, see the MLX release.
Evaluation
Language-model quality
WikiText-2 test-sample results:
| Model | Perplexity ↓ | Second-half perplexity ↓ |
|---|---|---|
| Qwen3.8-27B 4-bit | 7.7471 | 6.2370 |
| TernaryQuench 27B | 12.8569 | 10.0972 |
| Ternary-Bonsai-27B | 13.2475 | 10.1704 |
Protocol: the first 20,480 tokens of WikiText-2 test, split into 40 independent 512-token chunks. “Second half” scores the last 256 tokens of each chunk. TernaryQuench was measured directly from the Q2_K release file in llama.cpp; the reference rows used MLX. Raw Q2_K results include the file hash and per-chunk scores. Bonsai uses Qwen3.6 as its base.
Five-task zero-shot evaluation
Results measured directly from this Q2_K release file with the LM Evaluation Harness on the full task sets.
| Model | Mean accuracy ↑ | PIQA | ARC-Easy | ARC-Challenge | HellaSwag | WinoGrande |
|---|---|---|---|---|---|---|
| Qwen3.8-27B BF16 | 74.35 | 81.61 | 72.98 | 58.87 | 82.93 | 75.37 |
| TernaryQuench 27B | 72.28 | 78.78 | 78.96 | 55.29 | 73.96 | 74.43 |
| Ternary-Bonsai-27B | 71.91 | 79.27 | 76.22 | 55.63 | 75.98 | 72.45 |
Scores are percentages. The mean weights the five tasks equally, using length-normalized accuracy where available and standard accuracy for WinoGrande. Protocol: lm-eval 0.4.7, zero-shot, no chat template, full task sets. TernaryQuench uses llama.cpp CUDA with FP32 accumulation; reference rows use native PyTorch. Q2_K results and runtime provenance · Reference scores.
Runtime checks
The Q2_K file passed direct generation, JSON output without a grammar, and a complete tool-call round trip in Ollama 0.33.3 and llama.cpp. Its answers matched the original Q2_0 export on those short tests.
These checks verify installation and basic tool use. They do not establish reliability on complex, multi-turn agentic work.
Model
Ternary weights use three values per group: −scale, 0, +scale. This release retains higher precision in the final two decoder layers, embeddings, and output head:
| Component | Storage |
|---|---|
| Decoder layers 0–61 | 481 trained ternary matrices, packed as Q2_K |
| Decoder layers 62–63 | 15 BF16 matrices |
| Embeddings and output head | Q4_1 |
| Input | Text |
Q2_K stores the ternary weights in a format supported by standard Ollama and llama.cpp. Packing preserves every sign and zero but rounds group scales, introducing 1.55% relative RMS weight error across the converted matrices. All other tensors are unchanged. The conversion manifest records the error, tensor layout, file size, and SHA-256.
How it was trained
Starting from Qwen3.8-27B, we trained ternary weights with CAT-Q-style reconstruction and an AYOT-inspired calibration set: 512 sequences of 2,048 tokens, with 10% of rows drawn from agentic traces. The run took 55 hours 38 minutes on 1× NVIDIA H200 (141 GB).
The TernaryQuench build guide contains the recipe and commands for generating calibration traces, training, resuming a run, and exporting your own models.
License
Apache-2.0. This model is derived from Qwen3.8-27B, developed by Alibaba's Qwen team, under its Apache-2.0 license.