Diverse dialogue generation with context dependent dynamic loss function

Diverse dialogue generation with context dependent dynamic loss function
复制标题

DOI：
10.18653/v1/2020.coling-main.364
复制
发表时间：
2020-12
期刊：
--
影响因子：
--
通讯作者：
Ayaka Ueyama;Yoshinobu Kano
Ayaka Ueyama;Yoshinobu Kano
中科院分区：
其他
文献类型：
--
作者：
Ayaka Ueyama;Yoshinobu Kano

文献摘要

被引文献

相似文献

使用深度学习的对话系统已经实现了对用户话语的流畅响应句子的生成。然而，它们往往产生的反应并不多样化，也不太依赖于背景。为了解决这些缺点，我们提出了一个新的损失函数，一个逆N-gram损失（INF），它结合了上下文的流畅性和多样性在同一时间通过一个简单的公式。我们的INF损失可以通过使用应用于Softmax交叉熵损失的令牌的n-gram的逆频率的权重来动态调整其损失，以便在保持生成的句子的流畅性的同时更有可能出现稀有令牌。我们使用英语和日语的Twitter回复作为使用不同损失函数的单轮对话来训练Transformer。我们的INF损失模型在DIST-N和ROUGE等自动评估中优于SCE损失和ITF损失模型的基线，并且在我们的一致性和丰富性的人类评估中也获得了更高的分数。
Dialogue systems using deep learning have achieved generation of fluent response sentences to user utterances. Nevertheless, they tend to produce responses that are not diverse and which are less context-dependent. To address these shortcomings, we propose a new loss function, an Inverse N-gram loss (INF), which incorporates contextual fluency and diversity at the same time by a simple formula. Our INF loss can adjust its loss dynamically by a weight using the inverse frequency of the tokens’ n-gram applied to Softmax Cross-Entropy loss, so that rare tokens appear more likely while retaining the fluency of the generated sentences. We trained Transformer using English and Japanese Twitter replies as single-turn dialogues using different loss functions. Our INF loss model outperformed the baselines of SCE loss and ITF loss models in automatic evaluations such as DIST-N and ROUGE, and also achieved higher scores on our human evaluations of coherence and richness.