Estragon-9B
A Godot 4.7 / GDScript specialist. Qwen3.5-9B fine-tuned (SFT + GRPO with a headless-Godot judge as reward) to write correct, idiomatic, current-API GDScript — and nothing else you didn't ask for.
Godot the engine is named after Beckett's Waiting for Godot — the joke being that the engine would never be finished. Estragon is one of the two tramps who wait for the Godot who never arrives. This model is the reference-to-the-reference, with the inversion as the point: Estragon stopped waiting.
Benchmarks
Measured on GDScript-Eval v2 (gdeval_v2): 300 held-out tasks across 10 categories, judged by a headless Godot 4.7 binary — a task passes only if the generated file parses and its runtime behavior checks pass in-engine. Greedy decoding (pass@1), deployment system prompt. Harness, judge, and all task sets: https://github.com/blugart-dev/estragon.
| model | params | gdeval_v2 | % |
|---|---|---|---|
| ibm-granite/granite-4.1-8b | 8B | 123/300 | 41.0% |
| microsoft/phi-4 | 14B | 146/300 | 48.7% |
| google/gemma-4-12B-it (Q8 GGUF) | 12B | 211/300 | 70.3% |
| Qwen3-8B (previous lineage base) | 8B | 115/300 | 38.3% |
| Qwen3.5-9B (this model's base) | 9B | 219/300 | 73.0% |
| Estragon-9B (this model, bf16) | 9B | 248/300 | 82.7% |
| Claude Opus 4.8 (reference ceiling) | — | 300/300 | 100% |
All bases measured bare (no system prompt), greedy, same judge — the same protocol that produced the Qwen numbers. Gemma 4 measured via its official Q8_0 GGUF (its bf16 needs a transformers version incompatible with the pinned eval stack); Q8 is near-lossless but treat that row as ±a few tasks.
Sampling headroom: pass@8 = 285/300 — with 8 attempts the model solves 95% of the eval; a "generate 3, keep the first that parses in a headless Godot" wrapper recovers much of this cheaply.
Use it
The system prompt is part of the model. All numbers above were measured with it; without it the model is markedly weaker (prompt-inversion effect — the instruction tuning targets it):
You are Estragon, a Godot 4.7 GDScript specialist. Write correct, idiomatic
Godot 4 GDScript: static typing where practical, tab indentation, and current
Godot 4 APIs (@export/@onready annotations, Callable-based signal
connections, await, FileAccess, Time). Never use Godot 3 APIs or syntax.
Implement exactly what is asked: keep the requested names and signatures, and
do not add exported variables, extra methods, _ready blocks, print
statements, or demo code the task did not request. When asked for code,
respond with one complete GDScript file in a fenced code block, with brief
explanation only if asked.
Ollama
# with the Modelfile from the GGUF repo (bakes in the system prompt + params):
ollama create estragon -f Modelfile
ollama run estragon --think=false "A 2D platformer jump with coyote time, CharacterBody2D"
--think=false (API: "think": false) is required: Ollama's qwen3.5 runtime
manages thinking itself and a Modelfile cannot disable it (ollama#14617);
this is a non-thinking fine-tune (thinking mode measured −15 tasks).
llama.cpp
llama-server -m estragon-9b-Q4_K_M.gguf -ngl 99 --port 8080 \
--chat-template-kwargs '{"enable_thinking":false}'
# then chat with the system prompt above; temperature 0.7, top-p 0.8, top-k 20
⚠️ The enable_thinking:false flag is required. The embedded Qwen3.5
chat template otherwise opens a reasoning block; this is a non-thinking
fine-tune and will produce empty or degraded answers with thinking enabled
(measured: −15 tasks in thinking mode, and unclosed think blocks eat the
whole token budget). The Modelfile in this repo already handles it for
Ollama.
Q4_K_M runs fully GPU-offloaded on an 8 GB card (it was built on an RTX 3060 Ti). Recommended sampling: temperature 0.7, top_p 0.8, top_k 20 (Qwen3.5 non-thinking); the model is a non-thinking fine-tune — do not enable reasoning/thinking mode (measured: −15 tasks).
Quants
GGUF downloads: https://huggingface.co/Blugart/estragon-9b-gguf (this repo holds the bf16 weights). Every quant was run through the full 300-task eval (same judge, same protocol as the bf16 measurement standard) so the quantization cost is measured, not guessed:
| file | bits | size | gdeval_v2 | vs bf16 (248) |
|---|---|---|---|---|
| estragon-9b-Q4_K_M.gguf | 4-bit | 5.6 GB | 229/300 (76.3%) | −19 — the 8 GB VRAM pick; API-precision degrades most |
| estragon-9b-Q5_K_M.gguf | 5-bit | 6.5 GB | 249/300 (83.0%) | +1 — statistically identical to bf16; best size/quality trade |
| estragon-9b-Q8_0.gguf | 8-bit | 9.5 GB | 248/300 (82.7%) | ±0 — indistinguishable from bf16 |
Training
Solo project, ~$100 total compute budget (RunPod + Claude API for synthetic data). Pipeline: ~20k validated SFT pairs (docs-derived, real-repo-derived with permissive licenses, synthetic grounded in doc excerpts, mechanical Godot3→4 reverse-migration pairs — every example parse-validated by headless Godot 4.7 before entering the training set) → GRPO with the eval judge as reward (pass/fail + style-noise penalty + degeneracy detectors) on 254 runtime-verified training tasks (fully disjoint from the eval; shingle-level decontamination enforced). Full method, ADRs, harness, and eval sets: https://github.com/blugart-dev/estragon.
Limitations
- Godot 4.7 only. It will not write Godot 3; it may reject valid Godot 3 on principle. Untested against future 4.x API changes.
- The remaining 52/300 eval failures cluster in: rare-API precision (occasional hallucinated method names), long spec-compliance chains, and temporal semantics (timer/tween edge cases).
- GDScript only — no C#, no shader code beyond trivial cases.
- 9B params: it is a specialist, not a general assistant; out-of-domain quality is whatever survived from the base model.
License
Apache 2.0 (inherited from Qwen3.5-9B; fine-tuning data is original, doc-derived, or from permissively-licensed repos).