grug 27b v2
grug think small. code get stronger. draft head live in same rock.
grug start fresh from Qwen 3.8 27B. train for compact cave-think, coding, and tool use. Grug-tuned native MTP head already inside this checkpoint. no separate draft model to drag back to cave. every text rock in GGUF downloads carry head too.
v2 score better than grug v1.1 on measured code and tool hunts. session-title bug pass tested checks now. but grug not claim every bug dead: more effort not always more correct, hard thought sometimes leave cave-talk, and some loops still live. grug show gain and bruise below.
grug hunt code and tools
same H200/vLLM hunt rules for every model: temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05, seed 42. same task subsets. same output caps. code run against executable tests; tool calls face official AST checker. fixed subsets, not full leaderboard scores.
| Model | Medium HumanEval (132) | Medium MBPP (225) | Medium BFCL subset (200) |
|---|---|---|---|
| Qwen 3.8 27B | 99.2% | 93.3% | 89.0% |
| Grug v1.1 | 91.7% | 80.4% | 81.0% |
| Grug v2 | 97.0% | 90.7% | 88.0% |
grug also hunt real repo bugs: 12-issue SWE-bench Verified Django/SymPy subset. medium effort with original limits fix 10/12 for v2, 8/12 for v1.1. separate matched xhigh hunt keep reasoning in history, use 65,536-token context and 16,384 output tokens per tool turn: v2 fix 11/12, v1.1 7/12, Qwen 10/12.
this harness use shared pinned environment, not official per-instance Docker images. one bug swing score by 8.3 points. small hunt useful; not proof grug win everywhere.
session title should be title, not surprise tool call. v2 pass 90/90 session-title checks across low, medium, xhigh. zero tool-call leaks in those checks. real vLLM HTTP tests pass 24/24, including titles with tools present, streamed calls, thinking on and off. checks target reported title problem. grug did not test OpenCode desktop app itself.
keep reasoning in tool history and use original per-effort limits: v2 fix 9 / 10 / 11 of 12 at low/medium/xhigh. v1.1 fix 8 / 9 / 7. more effort help grug on this small agent hunt. not promise for every task.
base Qwen still hold more raw accuracy on some hunts. medium MATH, 150-case subset: v2 84.7%, Qwen 95.3%. compact thought cost something. grug not hide bruise.
all comparisons live in results.md: original and bigger output budgets, MTP accuracy, squeezed-rock smoke tests, repetition-penalty-1.0 reference runs. evaluation protocol and machine-readable results explain scoring corrections, completion rates, and limits. grug bring receipts.
grug turn think dial
pick low, medium, or xhigh with chat_template_kwargs.reasoning_effort. default medium. all three keep native effort instructions, all trained for terse Grug reasoning. inside voice grug; final answer and code can use ordinary language.
mean HumanEval think length at low/medium/xhigh:
- grug v2: 192 / 266 / 656 tokens
- Qwen: 483 / 617 / 1214
dial turn up, grug spend more thought on this code hunt. still fewer tokens than base brain.
but more thought not always more right. some coding and math scores fall at xhigh, even with double output limit. long traces sometimes drift into ordinary English. low-effort MATH audit still catch real unfinished loop. reported looping issue get mitigation, not complete cure.
saved template normalize old reasoning once. client still need correct reasoning and tool parsers. wrong plumbing make good brain stumble.
two heads, one rock
all 15 native draft-head tensors indexed in model-mtp.safetensors, right here in repo. during MTP tuning, only draft head learn; main brain stay frozen. held-out draft/verifier agreement climb 90.44% to 90.90%. distillation loss fall 0.3502 to 0.2965.
grug time 18 serial H200 prompts after warmup:
- no MTP: 68.02 tokens/sec
- one draft token: 112.77 tokens/sec
- two draft tokens: 142.44 tokens/sec
actual draft-token acceptance: 86.87% with one, 77.20% with two. speed numbers belong to this workload, not every cave computer. MTP-off/on greedy token sequences match on 10/18 prompts; not all paths say exact same tokens. separate coding/tool accuracy checks live in results.
head already packed. runtime still need switch turned on. grug test these vLLM 0.29.0 options:
vllm serve ProCreations/grug-27b-v2 \
--served-model-name grug-v2 --max-model-len 24576 \
--max-num-seqs 8 --gpu-memory-utilization 0.85 \
--reasoning-parser qwen3 --enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--speculative-config '{"method":"mtp","num_speculative_tokens":2}'
generation defaults: temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05. if client override model defaults, set these yourself. usage.md show requests, effort dial, history normalization, and llama.cpp commands.
how grug forge brain
main LoRA run use 3,466 training examples. validation split keep task groups apart. training mix carry verified code, math, repair material, inherited tool replay, and new title examples. best validation checkpoint merge into foundation brain.
vision encoder stay intact. grug not run new vision benchmark or prove quality across full configured context. eye still there; no new eye trophy claimed.
all model generation, training, merging, MTP tuning, quantization, runtime validation happen on HF Jobs. local GPU stay untouched. conservative whole-build estimate $120.92, including failed and canceled tries, below authorized $170 cap. estimate, not invoice. receipts: training.md, release provenance, HF Jobs cost ledger, SHA256SUMS.
Apache-2.0 license, same as foundation checkpoint. new benchmark test cases kept out of training mix. inherited replay and foundation pretraining may still hide overlap grug not recognize. training receipts and evaluation source included; raw inherited replay stay private.
grug made by ProCreations. base brain by Qwen team.