threadgrid-qwen35-4b-p4c (GGUF)

Qwen3.5-4B fine-tuned to read a screenshot of a grid path puzzle (a square grid with numbered waypoints and walls between cells) and reply with the board as JSON. Exported to GGUF f16 for Ollama.

It is the vision half of the side project thread-the-grid: the model reads the board, and an exact solver draws the route.

Inspired by the Zip puzzle games found online. The model was trained only on images drawn by the project's own renderer; no screenshots from any game were used for training.

Files

You need both GGUF files: the first is the language model, the second is the vision encoder. The adapter is only for people who want to merge or continue training themselves (PEFT 0.19.1, r=16, alpha=16, trained against unsloth/Qwen3.5-4B, a copy of the base model).

Use it with Ollama

Validated on Ollama 0.32.13 with an NVIDIA GPU (16 GB).

hf download Hero0963/threadgrid-qwen35-4b-p4c-gguf --local-dir threadgrid-qwen35-4b-p4c-gguf
cd threadgrid-qwen35-4b-p4c-gguf
ollama create threadgrid-qwen35-4b-p4c:f16 -f Modelfile
ollama show threadgrid-qwen35-4b-p4c:f16      # capabilities should list "vision"

Keep the Modelfile exactly as it is: it has no TEMPLATE line, and that is the setup that was validated. Pulling with ollama run hf.co/... directly is not equivalent, because it chooses a template of its own.

Send the image with this exact instruction, word for word (it still says "Zip" because that is the text the model was trained with) (prompt_variants.py):

Read this Zip puzzle screenshot and reply with ONLY a JSON object.
"layout" is a 2D array of two-character strings: "  " for an empty cell, "xx" for a blocked cell, and a zero-padded number such as "01" for a waypoint.
"walls" is a list of {"cell1": [row, col], "cell2": [row, col]} objects, one per thick black bar drawn on a grid line between two neighbouring cells. Report every wall you can see and do not invent any.

The project's app sends it to Ollama's OpenAI-compatible endpoint, /v1/chat/completions, with temperature 0, seed 42 and reasoning turned off (backends.py); the worked example below shows the exact request.

Worked example

One board, sent through the project's own reading path on 2026-09-19. It is a harder case than the training data: the project's editor draws walls in red, and every training image had black ones. One image shows that the model can do this, not that it always will.

Input: a 615 × 616 PNG, sent as it is (no resizing).

Request, as captured on the wire (the image's base64 is shortened here):

POST /v1/chat/completions
{
  "model": "threadgrid-qwen35-4b-p4c:f16",
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "<the instruction above, word for word>"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo... (38,038 chars)"}}
  ]}],
  "temperature": 0.0,
  "seed": 42,
  "reasoning_effort": "none",
  "stream": false
}

Reply, verbatim (6.7 s with the model already loaded):

{
  "layout": [
    ["  ", "02", "  ", "  ", "  ", "07"],
    ["  ", "01", "  ", "  ", "  ", "  "],
    ["  ", "  ", "  ", "05", "06", "  "],
    ["  ", "  ", "  ", "  ", "  ", "  "],
    ["  ", "03", "04", "  ", "  ", "  "],
    ["10", "  ", "  ", "  ", "09", "08"]
  ],
  "walls": [
    {"cell1": [0, 3], "cell2": [0, 4]},
    {"cell1": [2, 2], "cell2": [2, 3]},
    {"cell1": [2, 4], "cell2": [2, 5]},
    {"cell1": [3, 0], "cell2": [3, 1]},
    {"cell1": [3, 2], "cell2": [4, 2]}
  ]
}

Every waypoint and all five walls match the picture. An exact solver (CP-SAT) then finds the route through all 36 cells, and an independent checker confirms that it solves the board. The green numbers are the order of the steps:

The script that produced this, and its full output, are in the project repository under ai-collab/reports/artifacts/vlm-walkthrough/.

Run the whole app

The easiest way to run the whole thing, solver and UI included, is the project itself: python start.py from the repository, with these files placed as its model-weights guide describes.

How it was trained

Evaluation

Limitations

• It was trained on 6×6 boards from one renderer. It has learned that renderer's pictures, not screenshots in general: six real screenshots are far too few to claim more.
• The held-out score is already perfect, so it can no longer tell a better model from a worse one. A harder test set would be needed before changing anything.
• Always check the answer. The project's API reports solvable: false when the board it read has no solution, which means the reading was wrong.
• f16 needs about 9.1 GB of disk and a GPU with enough memory; it was not tested on CPU.

License

Apache-2.0, the same as the base model. This is a modified version of Qwen3.5-4B: a LoRA fine-tune merged into the weights and converted to GGUF. The base model's license text is in LICENSE.