Underdog-27B-1.0-Ternary
Underdog-27B by Conway is a 27-billion-parameter assistant for conversation, writing, coding, and summarization. This release combines fine-tuning and quantization-aware training with compact ternary weights for local inference. We finetuned this model to run better in Underdog for human users, based on Bonsai 2.
Choose GGUF for accelerated generation with Splash, or the MLX package for native Apple Silicon workflows.
Local performance
On an Apple M5 Max, Splash with DFlash2 delivered up to 157.9 generated tokens per second in our coding test.
| Task | MLX (bundled runtime) | GGUF (Splash + DFlash2) | Generation speedup |
|---|---|---|---|
| Email Writing | 42.0 tokens/s | 78.2 tokens/s | 1.86x |
| Code Generation | 41.0 tokens/s | 157.9 tokens/s | 3.85x |
| Long-Text Summarization | 40.4 tokens/s | 75.4 tokens/s | 1.87x |
Measured on Apple M5 Max with 40 GPU cores and 48 GiB unified memory. Both configurations use the same checkpoint in their respective MLX and GGUF packages. Values are means of two runs per task with temperature 0, thinking disabled, a 512-token output limit, and no reused prompt prefix. Engines were measured separately. Input lengths were 43, 44, and 3,003 tokens; generated output lengths may differ. Model loading and warmup are excluded. Splash 1.1.0 uses DFlash2 and INT8 KV; MLX 0.32.0 / MLX-LM 0.31.3 uses the bundled Hadamard-aware runtime without a draft model. The table reports generation throughput.
Run with Splash
On an Apple Silicon Mac, install Splash 1.1.0 or later, then run:
splash serve \
--model ConwayResearch/Underdog-27B-1.0-Ternary:PQ2_0 \
--language-only \
--default-reasoning-effort medium
The explicit :PQ2_0 selector selects the GGUF; Splash prepares its compatible draft model automatically. Open the chat page shown by the service, or connect your application through its OpenAI-compatible API. For medium thinking in the chat page, select Medium in the thinking menu.
Choose your model package
| Variant | Files to download | How to use |
|---|---|---|
| GGUF | Underdog-27B-PQ2_0.gguf and LICENSE | Load with Splash or a compatible PQ2_0 engine. |
| MLX | The complete mlx/ directory and LICENSE | Load the mlx/ directory with its included Hadamard-aware runtime. |
Download only the variant used by your application. Keep the MLX configuration, tokenizer, chat template, and runtime together with its weights.
hf download ConwayResearch/Underdog-27B-1.0-Ternary \
--include "mlx/*" "LICENSE" \
--local-dir ./underdog-27b-2bit
For Python integration, the supplied MLX runtime exposes vision_artifact.load_vl_model; use load_processor=False for text inference.