Qwen3-8B-UserRL-Agent
The UserRL baseline from MIMESIS: Qwen3-8B trained with multi-turn GRPO following UserRL, with GPT-5.5 as the training user simulator. The main text of the paper calls it GRPO w. GPT-5.5.
Paper · Code · Project page · Collection
Results
Mean task score of the five Qwen3-8B agents from the paper across eight UserRL environments, three of them held out from training (IntentionGym, TelepathyGym, SearchGym), under nine evaluation user models that were not used in training.
| Agent | Training user | Objective | Mean score |
|---|---|---|---|
| Base | none | SFT | 18.49 |
| UserRL (this model) | GPT-5.5 | GRPO | 26.10 |
| InfoPO | MIMESIS-9B | InfoPO | 26.65 |
| UserRL+ | MIMESIS-9B | GRPO | 29.54 |
| CSD | MIMESIS-9B | GRPO + CSD | 31.09 |
With the GRPO objective fixed, replacing GPT-5.5 with MIMESIS-9B as the training user (UserRL+) improves the mean score under every evaluation user. Each score is the mean over the nine evaluation users; per-user and per-environment scores are in Appendix D of the paper.
Training
RL starts from the Base checkpoint, Qwen3-8B after a supervised warm start, and trains on five UserRL environments: TravelGym, TurtleGym, FunctionGym, TauGym and PersuadeGym. The full configuration is in Appendix D of the paper, and the training scripts are in the agent directory of the code release.
Usage
The model loads like Qwen3-8B. To evaluate it in the eight environments, install the agent code, serve a user model as described there, and run from the agent directory:
bash scripts/run_eval.sh phanviethoang1512/Qwen3-8B-UserRL-Agent userrl
Citation
@article{phan2026mimesis,
title = {{MIMESIS}: Learning User Simulators as Training Environments for Interactive Agents},
author = {Phan, Hoang and Huynh, Dat and Zhmoginov, Andrey and Zeng, Qi and Mu, Wancen and Cao, Yue and Bi, Shengjie and He, Yun and Oh, Changdae and Lei, Deren},
journal = {arXiv preprint arXiv:2610.09484},
year = {2026}
}