III: Small: Collaborative Research: Scrutable and Explainable Information Retrieval with Model Intrinsic and Agnostic Approaches

信息检索系统是人们获取信息的重要手段。例如，智能搜索引擎广泛应用于基于Web的服务，如Web搜索、产品搜索和工作搜索。最近，复杂的数据和复杂的黑匣子模型使现代IR系统对用户的透明度降低。然而，随着越来越多的人依赖信息检索系统来指导他们的日常生活和决策，对于技术社区和普通公众来说，对可解释的搜索结果的需求越来越大，以便他们理解为什么提供某些搜索结果。与此同时，政府机构要求信息检索系统不仅提供高质量的结果，还要求提供合理的理由，以增强系统的可信性。该项目致力于开发算法和框架，以提高现代信息检索系统的可审查性、可解释性和透明度。它将促进大规模的学术和行业合作，通过促进可靠和可解释的信息访问服务的发展，使数十亿用户受益。该项目将为仔细审查和可解释的IR开发通用和可重复使用的框架。这一项目的研究将从两个方向进行。第一个方向是针对模型内在解释的新的检索模型。这包括为检索操作开发透明的推理过程和决策边界，支持用户反馈的结果探索的可仔细检查的功能，以及可跟踪的信息流以区分模型输入的贡献。第二个方向是建立模型不可知性解释的分析和模拟框架。这包括具有外部知识的事后解释系统，以及具有可解释输出的黑盒检索模型的模拟框架。除了模型固有的和模型不可知的方法，这个项目还将调查众包任务和系统指标，以比较内在和不可知性解释的有效性。研究成果将包括多个公共基准数据集和可解释IR的评估平台，这将为可持续和可重复的未来研究贡献研究社区。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Information Retrieval (IR) systems are important for people for information access. For example, intelligent search engines are widely used in Web-based services such as web search, product search, and job search. Recently, sophisticated data and complicated black-box models have made modern IR systems less transparent to users. However, as more and more people rely on IR systems to guide their daily life and decision making, there has been growing needs of explainable search results, both for technical communities and the general public, so that they understand why certain search results are provided. Meanwhile, governmental agencies are demanding IR systems to provide not only high-quality results, but also reasonable justifications, so as to enhance the trustworthiness of the systems. This project focuses on developing algorithms and frameworks to improve the scrutability, explainability, and transparency of modern IR systems. It will inspire large-scale academic-industry collaboration, which benefits billions of users by facilitating the development of reliable and explainable information access services. This project will develop general and reusable frameworks for scrutable and explainable IR. Research in this project will be performed on two directions. The first direction aims at new retrieval models for model-intrinsic explanation. This includes developing transparent inference process and decision boundaries for retrieval actions, scrutable functions that support result exploration with user feedback, and traceable information flow to distinguish the contribution of model inputs. The second direction aims at building analytical and simulative framework for model-agnostic explanation. This includes post-hoc explanation systems with external knowledge, and a simulation framework over black-box retrieval models with explainable outputs. Besides model-intrinsic and model-agnostic approaches, this project will also investigate crowd-sourcing tasks and systematic metrics to compare the effectiveness of intrinsic and agnostic explanations. The research outcomes will include multiple public benchmark datasets and evaluation platforms for explainable IR, which will contribute to the research community for sustainable and reproducible future studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.