BetterWright-4b — GGUF

Three quantisations (Q4_K_M, Q5_K_M, Q8_0) of ProCreations/BetterWright-4b.

GGUF conversion of ProCreations/BetterWright-4b — a full-parameter fine-tune of XHToken/Spark-X2.5-4B specialised for driving a real browser through BetterWright (a persistent, policy-guarded Playwright browser for AI agents).

Read the upstream card first. This repository is a format conversion only — no fine-tuning, no re-training, no behavioural changes. The architecture, benchmark results, training recipe and limitations are all inherited from the BF16 original. For the full story and benchmark caveats, see ProCreations/BetterWright-4b.

Available quantisations

Three quants of the same weights, all converted from the upstream BF16 release with llama-quantize (llama.cpp 0.5.0-dev, commit eae11d221):

All three carry the spark2_5 architecture, 290 tensors, the full 131,072-token vocabulary, and an embedded chat template. The embedding table and output head stay at the higher Q6_K in the K-quants.

Which to pick: Q5_K_M is the default balance and the one recommended unless you have a reason otherwise. Q8_0 is the closest to BF16 — use it when tool-call argument fidelity matters more than download size, or if you have VRAM to spare. Q4_K_M is for tight VRAM or bandwidth-limited setups; it still produces valid tool calls but is the most likely to drift on long, exacting browser code.

The upstream BF16 safetensors are 8.22 GB; these three are 32%, 36% and 53% of that respectively.

Requirements

llama.cpp ≥ build 3ad1ba7336 (2026-09-06, PR #27868 "[Model] Support for Spark2_5ForCausalLM"). Before that commit spark2_5 was an unknown architecture and loading failed.

docker pull ghcr.io/ggml-org/llama.cpp:server-vulkan   # verified working

The GGUF embeds its own chat template and tokenizer (131,072-token BPE vocab), so no extra files are needed.

Usage

llama.cpp server

Download the quant you want, then:

docker run --rm --gpus all --ipc=host -p 8080:8080 \
  -v /path/to/models:/models:ro \
  ghcr.io/ggml-org/llama.cpp:server-vulkan \
  -m /models/BetterWright-4b-Q5_K_M.gguf \
    -ngl 999 -fa on -c 262144 --host 0.0.0.0 --port 8080

# or, without a GPU
docker run --rm -p 8080:8080 \
  -v /path/to/models:/models:ro \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/BetterWright-4b-Q4_K_M.gguf -c 32768 --host 0.0.0.0 --port 8080

Or natively:

llama-server -m BetterWright-4b-Q5_K_M.gguf -ngl 999 -c 262144

llama.cpp CLI

llama-cli -m BetterWright-4b-Q8_0.gguf -ngl 999 -c 262144 \
  -p "What is 84 * 3 / 2?"

Drive a browser with BetterWright

npm install -g betterwright && betterwright setup
betterwright exec "find the top Hacker News story and give me its title and points" \
  --base-url http://127.0.0.1:8080/v1 --model betterwright-4b

Transformers / SGLang / vLLM

For BF16 accuracy and the reference benchmark numbers, use the upstream BF16 repo. Serving GGUF with vLLM requires its experimental GGUF loader and SGLang requires the Spark plugin; neither was used for the upstream numbers, so treat GGUF serving as unbenchmarked.

Tool calling and sampling

The GGUF preserves the Spark-X2.5 chat template, so llama.cpp's built-in parser handles tool calls natively — no manual regex or parser wiring needed.

• Tool format: <​tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></​tool_call>, thinking in <think>…</think>.
• Sampling defaults baked into the GGUF: temperature 1.0, top_p 0.95, top_k -1, repeat_penalty 1.0 — these match the upstream recommendation, so the defaults are already correct.
• Target harness: betterwright exec / the interactive betterwright console (tools browser, done, plus login/ask/handoff when the host exposes them). Also works from the BetterWright MCP server and the Pi extension.

Verified behaviour

All three quants were loaded in llama.cpp 0.5.0-dev (build 11277, commit eae11d221) on Vulkan with full GPU offload and exercised against the same prompts:

Responses differed slightly in phrasing across quants (expected), but all three produced well-formed tool calls with both required arguments. That is a functional smoke test, not a quality comparison — see the caveats below.

Architecture reference

From the GGUF metadata:

Recommended serving window: 262,144 tokens. The architecture's native positions extend to 1M, but the model was trained on sequences up to ~96k.

Memory at 262k context

KV cache dominates at long context: ~9.05 GiB in f16 at 262k tokens (9.0 GiB in the nine full-attention layers, only 0.05 GiB in the 27 sliding-window layers, since their window is only 512).

Use --cache-type-k q8_0 --cache-type-v q8_0 to cut the KV cache by about 25% (q8_0 is not half of f16 — it keeps a scale per 32-element block) if VRAM is tight. At 262k the KV cache is by far the largest term, so quantising the cache is usually a bigger lever than stepping down the model quant. At more typical contexts (8k–32k) the KV cache shrinks proportionally and all three quants fit comfortably in 6–8 GB.

Other quantisations

Only the three above are published here. To build others from the upstream BF16 repo:

# 1. convert the BF16 safetensors to GGUF (needs a llama.cpp build new enough for spark2_5)
python llama.cpp/convert_hf_to_gguf.py ProCreations/BetterWright-4b --outfile bw-f16.gguf --outtype f16

# 2. quantise
docker run --rm -v "$PWD":/work ghcr.io/ggml-org/llama.cpp:full \
  /work/bw-f16.gguf /work/BetterWright-4b-Q6_K.gguf Q6_K

Quality ordering, largest to smallest: BF16 → Q8_0 → Q6_K → Q5_K_M → Q5_K_S → Q4_K_M → Q4_0.

Notes and caveats

• This is a community conversion, not an official ProCreations release. The upstream project publishes BF16 safetensors; if you need an authoritative GGUF, please ask them. All three files were verified by SHA256 against the source and by live inference, but not perplexity-benchmarked.
• Quantisation is unbenchmarked for this model. The upstream benchmark (judge mean score 1.21 → 2.40, grounding 22% → 40%) was run on BF16. Each quant will differ somewhat, and tool-call argument accuracy is the most likely place to notice. We have not ranked these quants against each other on quality — treat Q5_K_M as a well-tested default, not a measured optimum. If fidelity to BF16 matters more than size, use Q8_0; it is the least lossy of the three.
• Benchmark numbers on the upstream card refer to BF16, not this file. Please don't quote them as GGUF results.
• Text-only: the model never sees screenshots; it relies on BetterWright's accessibility snapshots, DOM text and JSON results.
• Authenticated flows: trained on public, login-free sites and sandbox/demo sites for write actions; not trained on authenticated flows with real credentials.
• 4B parameters: strong at operating the browser, weaker than large models at open-ended reasoning about page content.

Acknowledgements

This is a format conversion of ProCreations/BetterWright-4b by the ProCreations / BetterWright Project. Base model by XHToken (Spark-X2.5-4B, Apache-2.0). Harness, benchmarks and browser: The BetterWright Project (MIT). Task sources: Online-Mind2Web (OSU NLP) and Odysseys (Jang et al., 2026, MIT-licensed task data).

The underlying weights, training recipe and benchmark results are the work of the upstream authors; this repository only repackages them in GGUF form for llama.cpp.