Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline
Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline
复制标题
DOI：
10.1007/978-3-030-72240-1_26
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Luyu Gao;Zhuyun Dai;Jamie Callan
中科院分区：
文献类型：
--
作者：
Luyu Gao;Zhuyun Dai;Jamie Callan
预训练的深度语言模型（LM）推进了文本检索的发展。从深度LM微调的重排序器基于丰富的上下文匹配信号来估计候选相关性。同时，深度LM也可以用来改善搜索索引，构建具有更好召回率的检索器。人们会期望两者在管道中的直接组合具有附加的性能增益。在本文中，我们发现，否则，流行的reranker不能充分利用改进的检索结果。因此，我们提出了一种用于训练rerankers的局部对比估计（LCE），并证明它显着改善了深度两阶段模型（我们的代码在 https://github.com/luyug/Reranker .).
Pre-trained deep language models (LM) have advanced the state-of-the-art of text retrieval. Rerankers fine-tuned from deep LM estimates candidate relevance based on rich contextualized matching signals. Meanwhile, deep LMs can also be leveraged to improve search index, building retrievers with better recall. One would expect a straightforward combination of both in a pipeline to have additive performance gain. In this paper, we discover otherwise and that popular reranker cannot fully exploit the improved retrieval result. We, therefore, propose a Localized Contrastive Estimation (LCE) for training rerankers and demonstrate it significantly improves deep two-stage models (Our codes are open sourced at https://github.com/luyug/Reranker .).