hybrid-routing-context-rl
The context-compression RL checkpoint (step 40) from the paper Screenshots or Tools? Eliciting Tool Us Context in Hybrid GUI–MCP Computer-UseAgents (arXiv:2608.03327).
A Qwen3-VL-8B-Thinking policy trained with multi-turn GRPO on OSWorld under a train/inference-consistent compressed observation rule: 2-image slip_success` (the screenshot after asuccessful MCP tool call is replaced by a text placeholder). Training and evaluation share the exact same observation policy — that matching is the
Results (OSWorld-MCP test_all_no_internet, 309 tasks, greedy ×3)
| accuracy | input cost | peak context (p95) | |
|---|---|---|---|
| Base operating point (img4, uncompressed) | 33.0% | 100% | 11.5K tok |
| This checkpoint (img2 + skip, compressed) | 37.8% | 53% | −37% |
Usage
This is a complete, directly-loadable Qwen3-VL-8B model directory. Deploy it with the compressed observation rule it was trained under — evaluating with the default rich observation underestimates it:
huggingface-cli download redai-infra/hybrid-routing-context-rl --local-dir ckpts/context_rl
MODEL=ckpts/context_rl CONTEXT_POLICY=skip_on_mcp_success MAX_IMAGE_HISTORY_LENGTH=2 \
bash scripts/run_mcp_eval.sh
Code, configs, and full provenance: https://github.com/redai-infra/hybrid-routing-agent — see results/context_rl/PROVENANCE.md for how every number above is derived, and configs/experiments/context_rl.yaml for the exact training recipe.
Citation
@article{fan2026screenshots,
title = {Screenshots or Tools? Eliciting Tool Use and Managing Multimodal
Context in Hybrid GUI-MCP Computer-Use Agents},
author = {Fan, Siqi and Li, Minghao and Ma, Xiaoqian and Tan, Wenhui and
Huang, Xiusheng and Wu, Juntong and Zhang, Liujie and Shang, Shuo
and Chen, Weihang},
journal = {arXiv preprint arXiv:2608.03327},
year = {2026},
url = {https://arxiv.org/abs/2608.03327}
}