Qwen3-VL-8B-Instruct-UI-Genie

An SFT fine-tuned reward model based on Qwen3-VL-8B-Instruct, trained on the UI-Genie-RM-517k dataset for GUI agent trajectory evaluation.

This model classifies GUI agent actions by generating a discrete preference token:

• <|+|> → action is correct (score = 1.0)
• <|-|> → action is wrong (score = 0.0)

A judge prompt is appended after the agent's tool-call response, and the model greedily decodes one classification token.

2026-08-10 update: this repo is now a self-contained merged checkpoint (LoRA fused + the separately-trained lm_head.weight baked in). An earlier upload shipped the LoRA adapter fused onto the untrained bootstrap lm_head.weight (the trainer writes the trained head to deepspeed checkpoint shards, not into adapter_model.safetensors, and a naive swift merge_lora silently keeps the bootstrap head — see lm_head_source.txt in this repo for provenance). That made the <|+|> / <|-|> logits nearly indistinguishable (row-norm diff ~3.8e-6 in the old upload vs. ~0.16 now). This upload uses the trained head and drops the LoRA-loading path below in favor of loading the model directly. Re-validated with rm_eval before publishing (see Evaluation below).

Prompt Format

The model was trained on a multi-turn format:

[system]    You are a helpful assistant. + mobile_use tool spec (with screen resolution)
[user]      The user query: <goal>
            Task progress (You have done the following operation on the current device):
            Step1: <action>  <image>
            ...
            StepN: <action>; <image>
[assistant] <tool_call>{"name": "mobile_use", "arguments": {...}}</tool_call>
[user]      Was the agent's action above correct given the current screen state?
            Answer with exactly one token: <|+|> for correct, <|-|> for wrong.
[assistant] ← model generates <|+|> or <|-|>

Intended Use

Use this model as a process reward model (PRM) to evaluate GUI agent actions on mobile UIs. It is well-suited for:

• Step-level correctness classification during agent rollouts
• Best-of-N action selection
• Filtering training data by action quality

Inference

Requirements

pip install vllm transformers torch pillow

Scoring with vLLM (merged checkpoint, no LoRA)

from vllm import LLM, SamplingParams
from transformers import AutoProcessor
from PIL import Image

MODEL_PATH = "Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie"

POS_TOKEN = "<|+|>"
NEG_TOKEN = "<|-|>"

JUDGE_PROMPT = (
    "Was the agent's action above correct given the current screen state? "
    "Answer with exactly one token: <|+|> for correct, <|-|> for wrong."
)

SYSTEM_PROMPT = (
    "You are a helpful assistant.\n\n# Tools\n\n"
    "You may call one or more functions to assist with the user query.\n\n"
    "You are provided with function signatures within <tools></tools> XML tags:\n"
    "<tools>\n"
    '{"type": "function", "function": {"name": "mobile_use", '
    '"description": "Use a touchscreen to interact with a mobile device. '
    "The screen's resolution is 540x1200.\", "
    '"parameters": {"properties": {"action": {"type": "string", '
    '"enum": ["click", "long_press", "swipe", "type", "key", "system_button", "open", "wait", "terminate"]}, '
    '"coordinate": {"type": "array"}, "text": {"type": "string"}, "button": {"type": "string"}}, '
    '"required": ["action"]}}}\n</tools>\n\n'
    "For each function call, return a json object within <tool_call></tool_call> XML tags:\n"
    "<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>"
)

# Load the merged model directly — no LoRA adapter needed.
llm = LLM(
    model=MODEL_PATH,
    dtype="bfloat16",
    gpu_memory_utilization=0.7,
    max_model_len=8192,
    limit_mm_per_prompt={"image": 10},
    enforce_eager=True,
)
processor = AutoProcessor.from_pretrained(MODEL_PATH, max_pixels=1_048_576)
tokenizer = processor.tokenizer

pos_id = tokenizer.encode(POS_TOKEN, add_special_tokens=False)[0]
neg_id = tokenizer.encode(NEG_TOKEN, add_special_tokens=False)[0]

sampling_params = SamplingParams(max_tokens=1, temperature=0.0)


def score_action(goal, prior_steps_text, tool_call_response, screenshot):
    """
    Score a GUI agent action.

    Args:
        goal:                 Task goal string.
        prior_steps_text:     String like "Step1: tap search\nStep2: type query\n"
        tool_call_response:   The agent's raw tool_call response string to judge.
        screenshot:           PIL.Image of the current screen state.

    Returns:
        float: 1.0 (correct), 0.0 (wrong), or 0.5 (undecided).
    """
    user_content = [
        {"type": "text", "text": f"The user query: {goal}\nTask progress (...): {prior_steps_text}; "},
        {"type": "image"},
    ]

    messages = [
        {"role": "system",    "content": SYSTEM_PROMPT},
        {"role": "user",      "content": user_content},
        {"role": "assistant", "content": tool_call_response.strip()},
        {"role": "user",      "content": JUDGE_PROMPT},
    ]

    prefix_text = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

    output = llm.generate(
        [{"prompt": prefix_text, "multi_modal_data": {"image": [screenshot]}}],
        sampling_params,
    )[0]

    gen_id = output.outputs[0].token_ids[0] if output.outputs[0].token_ids else None
    if gen_id == pos_id:
        return 1.0
    if gen_id == neg_id:
        return 0.0
    return 0.5   # neither token — treated as undecided


# Example
screenshot = Image.open("screenshot.png").convert("RGB")
tool_call  = '<tool_call>\n{"name": "mobile_use", "arguments": {"action": "click", "coordinate": [540, 120]}}\n</tool_call>'

score = score_action(
    goal="Tap the search button",
    prior_steps_text="Step1: opened the app\n",
    tool_call_response=tool_call,
    screenshot=screenshot,
)
print(f"Score: {score}")   # 1.0 = correct, 0.0 = wrong

Pairwise evaluation with rm_eval

python eval_rm.py \
    --rm_path Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie \
    --datasets ui-genie \
    --mode discrete \
    --uigenie_jsonl /path/to/reward_data_rm_pairs_last5.jsonl \
    --uigenie_images_dir /path/to/images \
    --output_dir results/

Evaluation

Per-side classification accuracy and pair accuracy (chosen classified correct AND rejected classified wrong) on held-out (chosen, rejected) pairs:

The AndroidFlux pair accuracy is low because the classifier is biased toward <|-|> on this out-of-domain, multi-agent replay data (chosen-side accuracy 29.6% vs. rejected-side 75.9%) — a known domain-gap limitation of this SFT stage, not an artifact of this upload's fix.

Training Details

Related Models

• Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring — Bradley-Terry scalar reward head built on top of this model.

Citation

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
}