EvoCUA-32B GGUF — quantized and verified against click-coordinate regression

GGUF conversions of meituan/EvoCUA-32B-20260105, a Qwen3-VL-based computer-use agent (CUA).

What makes this repo different: every quant here was measured on a click-coordinate regression suite, against a BF16 baseline of the same conversion. A CUA's output is a click coordinate, not prose — a broken preprocessing step yields perfectly fluent text with systematically shifted coordinates, and no perplexity number will tell you. Existing GGUF repos ship quants with no evidence either way. This one ships the evidence, the harness, and the raw per-case results.

👉 8B sibling repo, with a much wider quantization ladder (Q5 → IQ1) measured against the original transformers model: doldolggi/EvoCUA-8B-20260105-GGUF. Several conclusions below only hold because both sizes were measured — see Cross-size findings.

⚠️ Scope of the evidence. 16 grounding cases, English UI, single-turn, synthetic screenshots with constructed ground truth. No OSWorld end-to-end benchmark was run. Measured process/kernel noise floor is ~2 px — differences below that are not meaningful. Read the tables as "does quantization move the click", not as a task-success benchmark.

Results

Baseline is our own BF16 GGUF of this model (the 8B repo uses the actual transformers model, which is stronger; 32B BF16 is 61 GiB and does not fit a single 32 GB card).

Two metrics are reported separately because they mean different things: miss = emitted a coordinate that landed outside the target (grounding failure); non-click = chose wait/etc. so there is no coordinate (behavioural difference, not a failure).

All quants stay on-target 16/16 with no systematic offset. Degradation is monotonic but stays within ~2 px of BF16 across the whole range — i.e. quantization down to Q4 does not move the click in any way that matters at these target sizes.

Unsloth Dynamic (UD): what actually helps

The UD method has two parts — a custom imatrix and a per-tensor bit allocation. We separated them:

• The gain comes from the tensor allocation, not the imatrix. Swapping the imatrix changes little; swapping the recipe is what moves every metric. Confirmed on 8B with a cleaner control (imatrix held constant) — see the sibling repo.
• The imatrix source does not matter. Unsloth's (computed on the base model, before the CUA fine-tune) vs ours are indistinguishable. Using a base-model imatrix on a fine-tune is fine.
• UD-Q5_K_XL is the only build that reproduced BF16's action choice on all 16 cases, including a case where BF16 chose to wait before clicking a Delete button and stock Q5_K_M clicked instead.
• ⚠️ Tier-scoped. On 8B the UD advantage is clear at Q4–Q5 and vanishes at Q3/Q2. "UD is always better" is not what this data says.

Only one UD-Q5 build is uploaded (the Unsloth-imatrix one). The ablation arms above were measured, not shipped; their per-case JSON is in results/.

Cross-size findings (why the 8B repo matters)

• A 32B observation that UD hurt at Q4 (one 83 px outlier) failed to replicate on 8B, so we retracted it as n=1 noise. Single-size, single-case patterns are not patterns.
• The 8B ladder pushes down to IQ1, where grounding collapses completely (1/6 on-target, mean error half a screen) — with a stock-quant control showing it is the bit level, not the recipe. That tier doubles as a negative control: a harness where everything passes tells you nothing.

Files

BF16 (61 GiB) is not uploaded — regenerate in a few minutes:

python convert_hf_to_gguf.py <HF_DIR> --outfile evocua-32b-bf16.gguf --outtype bf16
python convert_hf_to_gguf.py <HF_DIR> --outfile mmproj-evocua-32b-f16.gguf --outtype f16 --mmproj

VRAM — read this before picking a quant

Measured on a 32 GB card (AMD R9700, 31.86 GiB usable):

⛔ Q6_K at 32K context does not fit a single 32 GB card, and llama.cpp will not tell you. It starts fine and answers correctly, ~9× slower: we measured the same file at 15.7 tok/s (-c 8192) vs 1.74 tok/s (-c 32768, VRAM saturated, degrading further as the KV cache grows). The only way to detect this is to measure decode speed.

Performance of the recommended build (1× 32 GB card, -c 32768, KV q8_0, -fa on, median of 3 restarts, one 1920×1080 screenshot in the prompt): prefill 17.4 tok/s, decode 17.6 tok/s, 28.8 GiB.

Serving

llama-server -m evocua-32b-UD-Q5_K_XL.gguf --mmproj mmproj-evocua-32b-f16.gguf \
  -ngl 99 -c 32768 --parallel 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 -fa on \
  --reasoning-format none \
  --host 127.0.0.1 --port 8080

Traps that cost us time — please read

1. --reasoning-format none is mandatory. With the default (auto), the thought block is moved into reasoning_content and the EvoCUA S2 parser — which reads <think> + <tool_call> out of content — silently sees nothing. No error, just an empty parse.
2. Never pass --reasoning-budget. Truncating the CoT breaks the agent loop.
3. 4K screenshots get downscaled, unlike upstream. llama.cpp's Qwen3-VL branch hardcodes set_limit_image_tokens(8, 4096) and neither writes nor reads image_min_pixels / image_max_pixels in the mmproj GGUF. Cap = 4096 merged tokens = 4.19 MP. 1920×1080 → 1920×1088 = 2040 tokens, no downscale. 3840×2160 = 8.3 MP is downscaled. Pass --image-max-tokens 16384 if you feed 4K. (Upstream EvoCUA allows 16.7 MP.)
4. Do not add --image-min-tokens 1024 just because llama.cpp warns about it, if you pre-resize your screenshots — it will upscale small images and diverge from upstream.

Coordinate convention (app side)

EvoCUA defaults to coordinate_type=relative: the system prompt states "The screen's resolution is 1000x1000" and the model emits a 0..999 grid. Absolute pixels = x * width / 999. Output is a tool_call, not pyautogui (that is the S1 prompt style):

<tool_call>{"name":"computer_use","arguments":{"action":"left_click","coordinate":[x,y]}}</tool_call>

Screenshots must be pre-resized with smart_resize(factor=32) by the caller, exactly as the upstream harness does.

How the verification works

harness/ contains everything. Two design decisions make the numbers trustworthy:

1. Test pages are rendered at 1920×1088 = 34 × 32, so smart_resize(factor=32) is the identity. Neither reference nor llama.cpp resizes the image, which removes preprocessing differences from the measurement and leaves only conversion/quantization.
2. Every click target is placed at absolute CSS coordinates, so the ground-truth bbox is a construction, not a hand measurement. make_testpages.py regenerates the whole suite.

Wide targets (e.g. a 1620 px file row) are reported separately: clicking the filename vs the row centre are both correct, so their large deltas say nothing about quantization. Judgement uses narrow targets (≤250 px) plus on-target rate.

A kernel-path control was run for this model: Q5_K_M re-measured with the exact multi-GPU + CPU-offload topology used for the BF16 baseline, versus its single-GPU result — 0.33 px mean, 2.0 px max, no systematic offset. That is the noise floor the tables above sit on.

Notes for AMD / ROCm (gfx1201, RDNA4) users

All measurements were made on 2× AMD Radeon AI PRO R9700 (gfx1201 / RDNA4, 32 GB each) under ROCm with a llama.cpp HIP build. gfx1201 numbers are scarce, so:

• The system ROCm install may not be enough to build. Ours had no hip / hip-lang / hipblas / rocblas CMake packages, so -DGGML_HIP=ON failed at configure. Point ROCM_PATH / CMAKE_PREFIX_PATH at a gfx1201-targeted ROCm SDK that has them (and set LD_LIBRARY_PATH at runtime).
• ⛔ Do not pass -ngl when splitting across multiple GPUs. It disables llama.cpp's automatic fitting: common_fit_params: ... n_gpu_layers already set by user ... abort. Manual values failed twice for us (CUBLAS_STATUS_ALLOC_FAILED on hipBLAS handle creation; cudaMalloc failed: out of memory). Passing only -mg <gpu> and leaving -ngl/-ts unset worked first try. llama.cpp knows the real buffer requirements; don't guess layers. This is how we ran the 61 GiB BF16 baseline across two cards: prefill 320 tok/s, decode 4.6 tok/s, 16 cases in ~9 minutes — versus a single-card -ngl 30 attempt that thrashed swap at 0.24 tok/s and never finished one case.
• ⚠️ If VRAM is short, llama.cpp starts fine and silently falls back to host memory (GTT). See the VRAM section above.

Reproducing

# 1) pull the UD tensor allocation out of Unsloth's published GGUF header (~12 MB, not 20 GB)
python harness/extract_ud_recipe.py \
  --repo unsloth/Qwen3-VL-32B-Thinking-GGUF \
  --file Qwen3-VL-32B-Thinking-UD-Q5_K_XL.gguf \
  --verify-against evocua-32b-bf16.gguf --out ud_q5_recipe.txt

# 2) quantize with it. token_embd / output need the dedicated flags, not --tensor-type.
#    Always --dry-run first: it validates the whole recipe in seconds.
llama-quantize --imatrix imatrix_unsloth.gguf \
  --tensor-type-file ud_q5_recipe.txt \
  --token-embedding-type q5_k --output-tensor-type q6_k \
  evocua-32b-bf16.gguf evocua-32b-UD-Q5_K_XL.gguf Q5_K_M 24

# 3) regenerate the test suite (deterministic: targets are CSS-positioned, not measured)
python harness/make_testpages.py

# 4) score a build against the baseline
python harness/coord_test.py --arm gguf --url http://127.0.0.1:8080/v1 --out coord_x.json
python harness/compare_arms.py --baseline results/coord_bf16_32b_2gpu.json --arms "x=coord_x.json"

Credits & licensing

• Base model: meituan/EvoCUA-32B-20260105 (Apache-2.0). These files are quantized derivatives of those weights.
• Unsloth Dynamic (UD) recipes are not ours. The UD-* files reproduce the per-tensor bit allocation published by Unsloth for the base model (unsloth/Qwen3-VL-32B-Thinking-GGUF). EvoCUA-32B is a fine-tune of that model, so the GGUF tensor names match exactly (707/707) and their allocation transfers. harness/extract_ud_recipe.py reads it out of the GGUF header via an HTTP range request. Their published imatrix is used where noted. Credit for the recipe belongs to Unsloth; the measurement is ours.
• Another 32B GGUF set exists at spicyneuron/meituan-EvoCUA-32B-20260105-GGUF. Ours are independent conversions; the numbers here describe these files.
• llama.cpp commit used: 3173a56471c1753650cd806694145ffd6dcace67 (ggml 0.22.0).
• Korean full report (covers 8B and 32B): REPORT.ko.md.