Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories
Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories
复制标题
DOI：
10.18653/v1/2021.emnlp-main.65
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
David Wilmot;Frank Keller
中科院分区：
文献类型：
--
作者：
David Wilmot;Frank Keller
测量事件显著性对于理解故事至关重要。本文采用了一种基于巴特基数函数和惊奇理论的无监督显著性检测方法，并将其应用于较长的叙事形式。我们改进了标准的Transformer语言模型，通过将外部知识库（来自检索增强生成），并添加了内存机制，以提高性能较长的作品。我们使用一种新的方法来获得显着性注释使用章节对齐的摘要从Shmoop语料库的经典文学作品。我们对这些数据的评估表明，我们的显着性检测模型提高了性能，超过了非知识库和记忆增强语言模型，这两者都是至关重要的。
Measuring event salience is essential in the understanding of stories. This paper takes a recent unsupervised method for salience detection derived from Barthes Cardinal Functions and theories of surprise and applies it to longer narrative forms. We improve the standard transformer language model by incorporating an external knowledgebase (derived from Retrieval Augmented Generation) and adding a memory mechanism to enhance performance on longer works. We use a novel approach to derive salience annotation using chapter-aligned summaries from the Shmoop corpus for classic literary works. Our evaluation against this data demonstrates that our salience detection model improves performance over and above a non-knowledgebase and memory augmented language model, both of which are crucial to this improvement.