Optimizing Guided Traversal for Fast Learned Sparse Retrieval
Optimizing Guided Traversal for Fast Learned Sparse Retrieval
复制标题
DOI：
10.1145/3543507.3583497
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Yifan Qiao;Yingrui Yang;Haixin Lin;Tao Yang
中科院分区：
文献类型：
--
作者：
Yifan Qiao;Yingrui Yang;Haixin Lin;Tao Yang
最近的研究表明，BM25驱动的动态索引跳过可以极大地提高基于MaxScore的文档检索的速度。本文研究了当使用其他模型如SPLADE和uniCOIL时，这种遍历指导策略在Top k检索中的有效性，发现当BM25模型与学习的权重模型不能很好地对齐或当检索深度k较小时，无约束的BM25驱动的跳过可能会有明显的相关性降级。本文在总结前人工作的基础上，采用两级剪枝控制机制和稀疏表示的模型对齐方法，对BM25引导索引遍历进行了优化，以实现快速检索。虽然可能会增加延迟的代价，但所提出的方案在保持相关性有效性的同时，比没有BM25指导的原始MaxScore方法要快得多。本文分析了该两级剪枝方案的竞争力，并在搜索多个测试数据集时对其在排序相关性和时间效率方面进行了权衡。
Recent studies show that BM25-driven dynamic index skipping can greatly accelerate MaxScore-based document retrieval based on the learned sparse representation derived by DeepImpact. This paper investigates the effectiveness of such a traversal guidance strategy during top k retrieval when using other models such as SPLADE and uniCOIL, and finds that unconstrained BM25-driven skipping could have a visible relevance degradation when the BM25 model is not well aligned with a learned weight model or when retrieval depth k is small. This paper generalizes the previous work and optimizes the BM25 guided index traversal with a two-level pruning control scheme and model alignment for fast retrieval using a sparse representation. Although there can be a cost of increased latency, the proposed scheme is much faster than the original MaxScore method without BM25 guidance while retaining the relevance effectiveness. This paper analyzes the competitiveness of this two-level pruning scheme, and evaluates its tradeoff in ranking relevance and time efficiency when searching several test datasets.