Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
复制标题
DOI：
10.48550/arxiv.2203.10249
复制
发表时间：
2022-03
期刊：
影响因子：
3
通讯作者：
Chao Zhao;Wenlin Yao;Dian Yu;Kaiqiang Song;Dong Yu;Jianshu Chen
中科院分区：
文献类型：
--
作者：
Chao Zhao;Wenlin Yao;Dian Yu;Kaiqiang Song;Dong Yu;Jianshu Chen
理解对话需要一个模型来捕捉话语中的各种关键信息，这些信息要么分散在周围，要么隐含在不同的对话回合中。因此，对话理解需要多种能力，如释义、总结和常识性推理。为了预训练零镜头对话理解模型，我们开发了一种新颖的叙事引导预训练策略，通过叙述对话输入的关键信息进行学习。然而，这种预训练策略的对话-叙述平行语料库目前还没有。为此，我们首先通过自动对齐电影字幕及其大纲来构建对话-叙事平行语料库。然后，我们在数据上预训练BART模型，并评估其在四个需要理解的基于对话的任务上的表现。实验结果表明，我们的模型不仅具有优异的零射击性能，而且具有更强的细粒度对话理解能力。数据和代码可在https://github.com/zhaochaocs/Diana上获得。
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations. Therefore, dialogue comprehension requires diverse capabilities such as paraphrasing, summarizing, and commonsense reasoning. Towards the objective of pre-training a zero-shot dialogue comprehension model, we develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input. However, the dialogue-narrative parallel corpus for such a pre-training strategy is currently unavailable. For this reason, we first construct a dialogue-narrative parallel corpus by automatically aligning movie subtitles and their synopses. We then pre-train a BART model on the data and evaluate its performance on four dialogue-based tasks that require comprehension. Experimental results show that our model not only achieves superior zero-shot performance but also exhibits stronger fine-grained dialogue comprehension capabilities. The data and code are available at https://github.com/zhaochaocs/Diana.