🌐 Website

📝 Paper

💻 Code

🚀 vLLM Serve (Recommended)

We recommend using vLLM for production deployment. Requires vllm>=0.12.0 with --trust-remote-code.

# OpenCUA-72B (8 GPUs, tensor parallel)
vllm serve xlangai/OpenCUA-72B \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --served-model-name opencua-72b \
  --host 0.0.0.0 \
  --port 8000

# OpenCUA-72B with data parallelism (tp=2, dp=4 for 4 instances on 8 GPUs)
vllm serve xlangai/OpenCUA-72B \
  --trust-remote-code \
  --tensor-parallel-size 2 \
  --data-parallel-size 4 \
  --gpu-memory-utilization 0.85 \
  --host 0.0.0.0 \
  --port 8000

Adjust --tensor-parallel-size, --data-parallel-size, and --gpu-memory-utilization based on your hardware configuration.

Introduction

OpenCUA models (OpenCUA-7B, OpenCUA-32B, and OpenCUA-72B) are end-to-end computer-use foundation models that can produce executable actions in the computer environments with great planning and grounding capabilities. They are based on the Qwen2.5-VL model family.

With the help of OpenCUA framework, our end-to-end agent models demonstrate strong performance across CUA benchmarks. In particular, OpenCUA-72B achieves an average success rate of 45.0% on OSWorld-Verified, establishing a new state-of-the-art (SOTA) among open-source models. OpenCUA-72B also has strong grounding ability, achieving 37.3% (SOTA) on UI-Vision and 60.8% on ScreenSpot-Pro.

📢 Updates

• 2026-01-17: 🎉 vLLM now fully supports OpenCUA-7B, OpenCUA-32B, and OpenCUA-72B! Thanks to the Meituan EvoCUA Team for their contributions to vLLM integration.

Key Features

• Superior Computer-Use Capablity: Able to execute multi-step computer-use actions with effective planning and reasoning
• Multi-OS Support: Trained on demonstrations across Ubuntu, Windows, and macOS
• Visual Grounding: Strong GUI element recognition and spatial reasoning capabilities
• Multi-Image Context: Processes up to 3 screenshot history for better context understanding
• Reflective Reasoning: Enhanced with reflective long Chain-of-Thought that identifies errors and provides corrective reasoning

Performance

Online Agent Evaluation

OpenCUA models achieves strong performance on OSWorld-Verified. OpenCUA-72B achieves the best performance among all open-source models with an average success rate of 45.0%, establishing a new state-of-the-art (SOTA).

OpenCUA scores are the mean of 3 independent runs.

GUI Grounding Performance

AgentNetBench (Offline Evaluation)

🚀 Quick Start

To align with our training infrastructure, we have modified the model in two places:

Installation & Download

First, install the required dependencies:

conda create -n opencua python=3.12
conda activate opencua
pip install openai>=1.0.0

Download the model weight from huggingface (optional, vLLM can download automatically):

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="xlangai/OpenCUA-72B",
    local_dir="OpenCUA-72B",
    local_dir_use_symlinks=False
)

🎯 GUI Grounding

First, start the vLLM server:

vllm serve xlangai/OpenCUA-72B \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --served-model-name opencua-72b \
  --host 0.0.0.0 \
  --port 8000

Then run the following code to test GUI grounding:

import base64
from openai import OpenAI

# vLLM server configuration
VLLM_BASE_URL = "http://localhost:8000/v1"
MODEL_NAME = "opencua-72b"  # Should match --served-model-name in vllm serve

def encode_image(image_path: str) -> str:
    """Encode image to base64 string."""
    with open(image_path, "rb") as f:
        return base64.b64encode(f.read()).decode()

def run_grounding(image_path: str, instruction: str) -> str:
    """Run GUI grounding inference via vLLM."""
    client = OpenAI(base_url=VLLM_BASE_URL, api_key="EMPTY")

    system_prompt = (
        "You are a GUI agent. You are given a task and a screenshot of the screen. "
        "You need to perform a series of pyautogui actions to complete the task."
    )

    messages = [
        {"role": "system", "content": system_prompt},
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{encode_image(image_path)}"}
                },
                {"type": "text", "text": instruction},
            ],
        },
    ]

    response = client.chat.completions.create(
        model=MODEL_NAME,
        messages=messages,
        max_tokens=512,
        temperature=0,
    )

    return response.choices[0].message.content

# Example usage
image_path = "screenshot.png"
instruction = "Click on the submit button"

result = run_grounding(image_path, instruction)
print("Model output:", result)

You can also run the grounding examples in OpenCUA/model/inference/:

cd ./model/inference/

# vLLM (requires running vLLM server first)
python vllm_inference.py

# HuggingFace Transformers
python huggingface_inference.py

🖥️ Computer Use Agent

OpenCUAAgent is developed in the OSWorld environment based on OpenCUA models. It iteratively perceives the environment via screenshots, produces reflective long CoT as inner monologue, and predicts the next action to be executed. OpenCUAAgent uses 3 images in total and L2 CoT format in default.

Command for running OpenCUA-72B in OSWorld:

    python run_multienv_opencua.py \
        --headless \
        --observation_type screenshot \
        --model OpenCUA-72B \
        --result_dir ./results --test_all_meta_path evaluation_examples/test_all_no_gdrive.json \
        --max_steps 100 \
        --num_envs 30  \
        --coordinate_type qwen25

Important Notes on Coordinate Systems

OpenCUA models output absolute coordinates after smart resize:

# Example output: pyautogui.click(x=960, y=324)
# These are coordinates on the smart-resized image, not the original image

# Convert to original image coordinates:
# Please refer to the smart_resize function in: https://github.com/huggingface/transformers/blob/67ddc82fbc7e52c6f42a395b4a6d278c55b77a39/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py#L55
def qwen25_smart_resize_to_absolute(model_x, model_y, original_width, original_height):
    # First, calculate the smart-resized dimensions
    resized_height, resized_width = smart_resize(original_height, original_width, factor = 28, min_pixels = 3136, max_pixels = 12845056)

    # Convert model output to relative coordinates on original image
    rel_x = model_x / resized_width
    rel_y = model_y / resized_height

    # Then convert to absolute coordinates on original image
    abs_x = int(rel_x * original_width)
    abs_y = int(rel_y * original_height)
    return abs_x, abs_y

Acknowledge

License

This project is licensed under the MIT License for Research and Commercial Use - see the LICENSE file in the root folder for details.

Citation

If you use OpenCUA models in your research, please cite our work:

@misc{wang2025opencuaopenfoundationscomputeruse,
      title={OpenCUA: Open Foundations for Computer-Use Agents},
      author={Xinyuan Wang and Bowen Wang and Dunjie Lu and Junlin Yang and Tianbao Xie and Junli Wang and Jiaqi Deng and Xiaole Guo and Yiheng Xu and Chen Henry Wu and Zhennan Shen and Zhuokai Li and Ryan Li and Xiaochuan Li and Junda Chen and Boyuan Zheng and Peihang Li and Fangyu Lei and Ruisheng Cao and Yeqiao Fu and Dongchan Shin and Martin Shin and Jiarui Hu and Yuyan Wang and Jixuan Chen and Yuxiao Ye and Danyang Zhang and Dikang Du and Hao Hu and Huarong Chen and Zaida Zhou and Haotian Yao and Ziwei Chen and Qizheng Gu and Yipu Wang and Heng Wang and Diyi Yang and Victor Zhong and Flood Sung and Y. Charles and Zhilin Yang and Tao Yu},
      year={2025},
      eprint={2508.09123},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2508.09123},
}