Having Your Cake and Eating it Too: Training Neural Retrieval for Language Inference without Losing Lexical Match
Having Your Cake and Eating it Too: Training Neural Retrieval for Language Inference without Losing Lexical Match
复制标题
鱼与熊掌兼得：在不丢失词汇匹配的情况下训练语言推理神经检索
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
M. Surdeanu
中科院分区：
文献类型：
--
作者：
Vikas Yadav;Steven Bethard;M. Surdeanu
我们提出了一个研究的重要性，信息检索（IR）技术的可解释性和性能的神经问答（QA）方法。我们发现，目前国家的最先进的Transformer方法（如RoBERTA）编码不好简单的信息检索（IR）的概念，如查询和文档之间的词汇重叠。为了减轻这一限制，我们引入了一种有监督的RoBERTa QA方法，该方法经过训练以模仿BM 25的行为和基于嵌入的对齐方法背后的软匹配思想。我们发现，融合简单的词汇匹配IR概念的Transformer技术的结果在改善a）的（词汇匹配）的可解释性，B）检索性能，和c）的QA性能在两个多跳QA数据集。我们进一步强调的词汇鸿沟的差距桥接能力的Transformer方法通过分析上下文与词汇匹配的令牌对的监督Roberta分类器的注意力分布。
We present a study on the importance of information retrieval (IR) techniques for both the interpretability and the performance of neural question answering (QA) methods. We show that the current state-of-the-art transformer methods (like RoBERTa) encode poorly simple information retrieval (IR) concepts such as lexical overlap between query and the document. To mitigate this limitation, we introduce a supervised RoBERTa QA method that is trained to mimic the behavior of BM25 and the soft-matching idea behind embedding-based alignment methods. We show that fusing the simple lexical-matching IR concepts in transformer techniques results in improvement a) of their (lexical-matching) interpretability, b) retrieval performance, and c) the QA performance on two multi-hop QA datasets. We further highlight the lexical-chasm gap bridging capabilities of transformer methods by analyzing the attention distributions of the supervised RoBERTa classifier over the context versus lexically-matched token pairs.