RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
复制标题
DOI：
10.18653/v1/2021.naacl-main.100
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Srini Iyer;Sewon Min;Yashar Mehdad;Wen-tau Yih
中科院分区：
文献类型：
--
作者：
Srini Iyer;Sewon Min;Yashar Mehdad;Wen-tau Yih
用于开放域问题分类（QA）的最先进的机器阅读理解（MRC）模型通常使用远距离监督的正面示例和启发式检索的负面示例来训练跨度选择。这种训练方案可能解释了经验观察，即这些模型在其前几个预测中实现了高召回率，但总体准确率较低，从而激发了对答案重新排名的需求。我们开发了一种成功的重新排序方法（RECONSIDER），用于跨度提取任务，该任务提高了MRC模型的性能，甚至超出了大规模的预训练。RECONSIDER基于从高置信度MRC模型预测中提取的正面和负面示例进行训练，并使用通道内跨度注释在较小的候选集上执行以跨度为中心的重新排名。因此，RECONSIDER学习消除接近的误报，在四个QA任务上实现了新的提取技术，在自然问题与真实的用户问题上的精确匹配准确率为45.5%，在TriviaQA上为61.7%。我们将发布所有相关数据、模型和代码。
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples. This training scheme possibly explains empirical observations that these models achieve a high recall amongst their top few predictions, but a low overall accuracy, motivating the need for answer re-ranking. We develop a successful re-ranking approach (RECONSIDER) for span-extraction tasks that improves upon the performance of MRC models, even beyond large-scale pre-training. RECONSIDER is trained on positive and negative examples extracted from high confidence MRC model predictions, and uses in-passage span annotations to perform span-focused re-ranking over a smaller candidate set. As a result, RECONSIDER learns to eliminate close false positives, achieving a new extractive state of the art on four QA tasks, with 45.5% Exact Match accuracy on Natural Questions with real user questions, and 61.7% on TriviaQA. We will release all related data, models, and code.