Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation
复制标题
DOI：
10.18653/v1/2021.findings-acl.338
复制
发表时间：
2021-06
影响因子：
1.3
通讯作者：
Prakhar Gupta;Yulia Tsvetkov;Jeffrey P. Bigham
中科院分区：
文献类型：
--
作者：
Prakhar Gupta;Yulia Tsvetkov;Jeffrey P. Bigham
开放域神经对话模型在响应排名和评估任务方面已经达到了高性能。这些任务被称为对话环境中给出的响应的二进制分类，并且模型通常学会根据上下文响应内容相似性进行预测。但是，对内容相似性的过度依赖使模型对不一致的存在，不正确的时间表达式和其他对响应适当性和连贯性重要的因素敏感。我们提出了自动创建对抗性负面培训数据的方法，以帮助排名和评估模型学习超出内容相似性的功能。我们提出了面具填充和关键字引导的方法，以产生负面示例，以培训更强大的对话系统。这些产生的对抗反应与上下文具有很高的内容相似性，但不连贯，不合适或不流利。我们的方法是完全数据驱动的，可以轻松地将其纳入现有模型和数据集中。关于多个数据集的分类，排名和评估任务的实验表明，我们的方法在为培训对话系统提供信息的负面示例方面表现优于强大的基准。
Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks. These tasks are formulated as a binary classification of responses given in a dialogue context, and models generally learn to make predictions based on context-response content similarity. However, over-reliance on content similarity makes the models less sensitive to the presence of inconsistencies, incorrect time expressions and other factors important for response appropriateness and coherence. We propose approaches for automatically creating adversarial negative training data to help ranking and evaluation models learn features beyond content similarity. We propose mask-and-fill and keyword-guided approaches that generate negative examples for training more robust dialogue systems. These generated adversarial responses have high content similarity with the contexts but are either incoherent, inappropriate or not fluent. Our approaches are fully data-driven and can be easily incorporated in existing models and datasets. Experiments on classification, ranking and evaluation tasks across multiple datasets demonstrate that our approaches outperform strong baselines in providing informative negative examples for training dialogue systems.