Boulesis-26B-A4B — GGUF (static quants)
[!Note] All credit for the model goes to SubMaroon, who produced Boulesis-26B-A4B — an experimental Gemma-4-26B-A4B RP merge that splices attention circuits (QK routing from Pantheon-Reasoning-1.1, a transplanted style head from StyleTune-V2, a narrative RP LoRA) onto the coder3101 heretic base. These are quantizations only.
[!Tip] Benchmarked on CaliperBench — a creative-writing benchmark scoring prose craft, roleplay and willingness rather than general intelligence. See this model's scores: caliperbench.com
[!Important] These GGUFs embed the fixed Gemma-4 chat template. The author's shipped template omits the thought-channel prime on the thinking path, so thinking mode returns empty content on stock llama.cpp; the embedded one is corrected. Serve with
--jinjaand pass--no-reasoning-preservefor multi-turn thinking, or the model re-reads its own earlier thoughts and answers with reasoning only.
Provided quants
| Quant | Size | Notes |
|---|---|---|
| Q4_K_M | 17.2 GB | fast, recommended |
| Q6_K | 23.2 GB | very good quality |
| Q8_0 | 27.6 GB | fast, best quality |
Loading (any current llama.cpp)
llama-server -m Boulesis-26B-A4B.Q6_K.gguf -ngl 99 -fa on -c 32768 -ctk q8_0 -ctv q8_0 --jinja --no-reasoning-preserve
Thinking: on by default. Turn off per request with chat_template_kwargs: {"enable_thinking": false} or serve with --reasoning off --reasoning-budget 0. With thinking on, cap it per request with thinking_budget_tokens (1536 is plenty for roleplay).
Quant recipe
bf16 GGUF from the safetensors via convert_hf_to_gguf.py (llama.cpp b10819); the author's repo ships a placeholder tokenizer.json, so the base google/gemma-4-26B-A4B-it tokenizer (shared by the whole 26B-A4B family) was used for the conversion. Plain llama-quantize from that bf16 — no requantizing from a lower quant. The i1 set uses an imatrix computed on the same bf16 over bartowski calibration_datav3.
License
Gemma Terms of Use, inherited from the base model.