VISTA-9B
VISTA-9B are GUI-grounding vision-language models trained from Qwen3.5 9B backbones with VISTA: View-Consistent Self-Verified Training for GUI Grounding.
Model Description
VISTA-9B is a GUI-grounding model that maps a screenshot and a natural-language instruction to a click coordinate in the normalized 0-1000 image frame.
- View-consistent GRPO training. VISTA builds each GRPO comparison group from target-preserving views of the same GUI instance, with exact coordinate remapping across cropped views. This exposes localization behavior under semantically equivalent but geometrically different screenshots.
- Self-verified cross-view anchoring. The training objective adds an oracle-format center-point anchor only when model-generated rollouts have already produced a maximum-reward prediction, stabilizing short coordinate generation without unconditional imitation on all-fail groups.
Evaluation
Accuracy is reported for GUI grounding. The model predicts a normalized coordinate in the 0-1000 frame, and the prediction is counted as correct if the point lies inside the target element. All reported results use deterministic decoding at temperature 0 and single-view inference.
Results on GUI Grounding benchmarks
| Model | SSPro | SSV2 | OSWorld-G | OSWorld-G-R |
|---|---|---|---|---|
| Qwen3.5-4B | 60.3 | 90.4 | 54.4 | 66.8 |
| GRPO-4B | 62.2 | 94.2 | 59.9 | 69.2 |
| VISTA-4B | 64.2 | 93.8 | 61.2 | 69.7 |
| Δ | +2.0 | -0.4 | +1.3 | +0.5 |
| Qwen3.5-9B | 65.2 | 91.9 | 63.1 | 74.6 |
| GRPO-9B | 68.3 | 95.2 | 67.5 | 75.2 |
| VISTA-9B | 69.2 | 95.8 | 68.1 | 75.5 |
| Δ | +0.9 | +0.6 | +0.6 | +0.3 |
| Qwen3.5-35B-A3B | 68.6 | 93.8 | 65.8 | 72.5 |
| GRPO-35B-A3B | 71.7 | 95.7 | 70.4 | 74.3 |
| VISTA-35B-A3B | 72.9 | 95.8 | 71.5 | 75.3 |
| Δ | +1.2 | +0.1 | +1.1 | +1.0 |
Quick Start
Use the same image-chat interface as the underlying Qwen3.5 vision-language model. The recommended prompt is:
Output the center point of the position corresponding to the instruction: {instruction}. The output should just be the coordinates of a point, in the format [x,y].
Example:
import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "inclusionAI/VISTA-9B"
model = AutoModelForImageTextToText.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
image = Image.open("screenshot.png").convert("RGB")
instruction = "Click the search button"
prompt = (
"Output the center point of the position corresponding to the instruction: "
f"{instruction}. The output should just be the coordinates of a point, "
"in the format [x,y]."
)
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": prompt},
],
}
]
text = processor.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = processor(
text=[text],
images=[image],
padding=True,
return_tensors="pt",
).to(model.device)
generated = model.generate(
**inputs,
max_new_tokens=32,
do_sample=False,
)
new_tokens = generated[:, inputs.input_ids.shape[1]:]
response = processor.batch_decode(new_tokens, skip_special_tokens=True)[0].strip()
print(response) # e.g. [512,384]
Citation
Please consider citing if you find our work useful:
@misc{qiu2026vista,
title={VISTA: View-Consistent Self-Verified Training for GUI Grounding},
author={Xinyu Qiu, Yunzhu Zhang, Heng Jia, Shuheng Shen, Changhua Meng, Linchao Zhu},
year={2026},
eprint={2606.14579},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2606.14579},
}