A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agents
A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agents
复制标题
DOI：
10.18653/v1/w19-5942
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
A. C. Curry;Verena Rieser
中科院分区：
文献类型：
--
作者：
A. C. Curry;Verena Rieser
会话代理人应该如何通过用户对言语虐待做出反应？为了回答这个问题，我们对当前最先进的系统所采用的滥用响应策略进行了大规模的众包评估。我们的研究结果表明，一些策略，如“礼貌拒绝”，得分很高，而对于其他策略的人口因素，如年龄，以及严重程度的前虐待影响用户的看法，其中的反应是适当的。此外，我们发现，大多数数据驱动的模型落后于基于规则的或商业系统在其感知的适当性。
How should conversational agents respond to verbal abuse through the user? To answer this question, we conduct a large-scale crowd-sourced evaluation of abuse response strategies employed by current state-of-the-art systems. Our results show that some strategies, such as “polite refusal”, score highly across the board, while for other strategies demographic factors, such as age, as well as the severity of the preceding abuse influence the user’s perception of which response is appropriate. In addition, we find that most data-driven models lag behind rule-based or commercial systems in terms of their perceived appropriateness.