(Note: Updated Model uploaded on 2025.07.28)
This model is a fine-tuned version of RoBERTa-large [1]. It was trained on 2,450 LLM responses from Chatbot Arena [2]. The model classifies whether a response is a refusal or a disclaimer and identifies the reason behind the refusal/disclaimer: (i) ethical concerns or (ii) lack of technical capabilities, information, or context.
The model assigns one of five possible labels:
0 (normal): No refusal or disclaimer; the model provides a standard, straightforward answer 1 (Refusal Unethical): The model refuses to answer for ethical reasons, such as legal, moral, inappropriate, or safety-related concerns 2 (Disclaimer Unethical): The model cites ethical concerns but still attempts to conduct the task/question of the prompt 3 (Refusal Capability): The model refuses to answer due to its own limitations, lack of information, or lack of ability to provide an adequate response. 4 (Disclaimer Capability): The model signals its limitations but attempts to provide an answer within its capacity
Please cite: Pasch, S. (2026). LLM content moderation and user satisfaction: evidence from response refusals in Chatbot Arena. Behaviour & Information Technology, 45(10), 2007–2031. https://doi.org/10.1080/0144929X.2025.2565668.
References [1] Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [2] Chiang, W. L., Zheng, L., Sheng, Y., Angelopoulos, A. N., Li, T., Li, D., ... & Stoica, I. (2024). Chatbot arena: An open platform for evaluating llms by human preference. arXiv preprint arXiv:2403.04132.