WDRASS: A Web-scale Dataset for Document Retrieval and Answer Sentence Selection
WDRASS: A Web-scale Dataset for Document Retrieval and Answer Sentence Selection
复制标题
WDRASS：用于文档检索和答案句子选择的网络规模数据集
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Alessandro Moschitti
中科院分区：
文献类型：
--
作者：
Zeyu Zhang;Thuy Vu;S. Gandhi;Ankit Chadha;Alessandro Moschitti
开放域问答（ODQA）系统从搜索引擎返回的相关文本生成答案，例如，基于词汇特征的BM25，或基于嵌入的密集通道检索（DPR）。很少有数据集可用于此任务：它们主要关注基于机器阅读（MR）方法的QA系统，并显示有问题的评估，主要基于非上下文化的简短答案匹配。在本文中，我们提出了一个基于答案句子选择（AS2）模型的ODQA数据集WDRASS，该模型将句子作为QA系统的候选答案。WDRASS由约64k个问题和800k多个标记段落和句子组成，这些段落和句子从30M个文档中提取出来。我们通过在数据集上训练模型并与谷歌NQ上训练的相同模型进行比较来评估数据集。我们的实验表明，WDRASS显著提高了检索和重新排序模型的性能，从而提高了下游QA任务的准确性。我们相信我们的数据集可以在推进IR研究方面产生重大影响。
Open-Domain Question Answering (ODQA) systems generate answers from relevant text returned by search engines, e.g., lexical features-based such as BM25, or embeddings-based such as dense passage retrieval (DPR). Few datasets are available for this task: they mainly focus on QA systems based on machine reading (MR) approach, and show problematic evaluation, mostly based on uncontextualized short answer matching. In this paper, we present WDRASS, a dataset for ODQA based on answer sentence selection (AS2) models, which consider sentences as candidate answers for QA systems. WDRASS consists of ∼64k questions and 800k+ labeled passages and sentences extracted from 30M documents. We evaluate the dataset by training models on it and comparing with the same models trained on Google NQ. Our experiments show that WDRASS significantly improves the performance of retrieval and reranking models, thus boosting the accuracy of downstream QA tasks. We believe our dataset can produce significant impact in advancing IR research.