🔍 YOLOv11l — UI Groups Detection
This model is a fine-tuned version of Ultralytics/YOLO11, trained to detect UI groups (e.g., toolbars, tab groups) in macOS application screenshots.
It is part of the Screen2AX project, a research-driven effort to generate accessibility metadata for macOS applications using vision-based techniques.
⚠️ AGPL-3.0 licence — read before integrating. This model inherits the AGPL-3.0 licence from Ultralytics YOLO11. Embedding it in a product or service — including serving it behind a network API — requires open-sourcing the integrating application under AGPL-3.0, or obtaining a commercial Ultralytics Enterprise Licence. Details in License.
🧠 Task Overview
- Task: Object Detection
- Target: macOS UI groups
- Supported Label(s):
['AXGroup']
This model detects higher-level UI groupings that are commonly used to structure accessible interfaces (e.g., AXGroup, AXTabGroup, AXToolbar, etc.).
🗂 Dataset
- Training data:
macpaw-research/Screen2AX-Group
🚀 How to Use
🔧 Install Dependencies
pip install huggingface_hub ultralytics
🧪 Load the Model and Run Predictions
from huggingface_hub import hf_hub_download
from ultralytics import YOLO
# Download the model from the Hugging Face Hub
model_path = hf_hub_download(
repo_id="macpaw-research/yolov11l-ui-groups-detection",
filename="ui-groups-detection.pt"
)
# Load and run prediction
model = YOLO(model_path)
results = model.predict("/path/to/your/image")
# Visualize or process results
results[0].show()
📜 License
This model is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0), inherited from the original YOLOv11 base model and the Ultralytics training framework.
What this means in practice:
- ✅ Research, evaluation, and open-source (AGPL) projects — free to use, modify, and redistribute, provided derivative works are also released under AGPL-3.0.
- ⚠️ Commercial or closed-source use — AGPL-3.0 is a strong copyleft licence whose obligations also trigger on network use: shipping this model inside a proprietary application, or exposing it through a hosted API, requires releasing the integrating application's source code under AGPL-3.0.
- 💼 Need a proprietary integration? Ultralytics offers a commercial Enterprise Licence that lifts the AGPL obligations for YOLO models and their derivatives, including fine-tunes such as this one.
This summary is provided for convenience and is not legal advice.
🔗 Related Projects
✍️ Citation
If you use this model, please cite the Screen2AX paper:
@misc{muryn2025screen2axvisionbasedapproachautomatic,
title={Screen2AX: Vision-Based Approach for Automatic macOS Accessibility Generation},
author={Viktor Muryn and Marta Sumyk and Mariya Hirna and Sofiya Garkot and Maksym Shamrai},
year={2025},
eprint={2507.16704},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2507.16704},
}
🌐 MacPaw Research
Learn more at https://research.macpaw.com