III: Small: Efficiency Optimization for Neural Document Ranking with Compact Representations

III：小：具有紧凑表示的神经文档排序的效率优化

• 批准号: 2225942
• 负责人: Tao Yang
• 金额: $ 59.37万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2225942&HistoricalAwards=false
• 关键词: III; Small; Efficiency; Optimization; Neural

Over the last few years, the resurgence of neural models has greatly advanced the field of information retrieval enabling retrieval engines to effectively match and rank search results in response to a user query. For example, this new technology has enable to determine the most relevant documents in response to a query even when some query keywords may not appear in these documents. The main drawback of using deep neural models for ranking is that the retrieval is extremely time consuming. As a result, such models cannot be deployed in many practical search applications. This project is focused on studying efficient solutions to perform neural ranking computation and the developed techniques will be evaluated using public datasets to assess the solution’s effectiveness. The project integrates the research with an educational plan including undergraduate and graduate students' involvement, instructional material development, and outreach activities. This project carries out a two-thrust research agenda for efficient neural ranking. The first thrust investigates a fast re-ranking scheme for a dual-encoding architecture by leveraging precomputed embeddings to compose a query representation with approximation, and combining deep contextual token interactions and traditional lexical matching features. The second thrust of this project investigates a compact representation of document embeddings and strike a balance of relevance and space efficiency which affects online inference latency. The project exploits the composite nature of ranking inference for answering a query to approximate query embeddings, and decouples ranking contribution of document embeddings in deriving a compact representation. This research will advance our fundamental understanding of relevance and efficiency tradeoffs in neural information retrieval, and significantly reduce the computing and space cost of online inference while retaining the essential benefits of deep learning for effective ranking on affordable computing platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的几年里，神经模型的复兴极大地推进了信息检索领域，使检索引擎能够有效地匹配和排名搜索结果以响应用户查询。例如，这种新技术使得能够响应于查询来确定最相关的文档，即使某些查询关键字可能没有出现在这些文档中。 使用深度神经模型进行排名的主要缺点是检索非常耗时。因此，这样的模型不能部署在许多实际的搜索应用程序。 该项目的重点是研究执行神经排序计算的有效解决方案，并将使用公共数据集评估所开发的技术，以评估解决方案的有效性。该项目将研究与教育计划相结合，包括本科生和研究生的参与，教学材料的开发和推广活动。该项目针对高效神经排名实施了两个重点的研究议程。第一个推力研究了一个快速重新排序方案的双重编码架构，利用预先计算的嵌入组成的查询表示与近似，并结合深上下文令牌的相互作用和传统的词汇匹配功能。该项目的第二个重点是研究文档嵌入的紧凑表示，并在影响在线推理延迟的相关性和空间效率之间取得平衡。该项目利用排序推理的复合性质来回答近似查询嵌入的查询，并在导出紧凑表示时对文档嵌入的排序贡献进行简化。这项研究将促进我们对神经信息检索中相关性和效率权衡的基本理解，并显著降低在线推理的计算和空间成本，同时保留深度学习的基本优势，以便在负担得起的计算平台上进行有效排名。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响进行评估，被认为值得支持审查标准。

项目成果

Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval

• DOI: 10.1145/3539618.3592051
• 发表时间: 2023-06
• 期刊: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
• 作者: Yifan Qiao;Yingrui Yang;Shanxiu He;Tao Yang

Optimizing Guided Traversal for Fast Learned Sparse Retrieval

• DOI: 10.1145/3543507.3583497
• 发表时间: 2023-04
• 期刊: Proceedings of the ACM Web Conference 2023
• 作者: Yifan Qiao;Yingrui Yang;Haixin Lin;Tao Yang

Balanced Knowledge Distillation with Contrastive Learning for Document Re-ranking

• DOI: 10.1145/3578337.3605120
• 发表时间: 2023-08
• 期刊: Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval
• 作者: Yingrui Yang;Shanxiu He;Yifan Qiao;Wentai Xie;Tao Yang