III: Small: Collaborative Research: Scrutable and Explainable Information Retrieval with Model Intrinsic and Agnostic Approaches

信息检索系统是人们获取信息的重要手段。例如，智能搜索引擎广泛应用于基于web的服务，如web搜索、产品搜索、工作搜索等。最近，复杂的数据和复杂的黑匣子模型使得现代红外系统对用户来说不那么透明。然而，随着越来越多的人依赖IR系统来指导他们的日常生活和决策，对于技术社区和一般公众来说，对可解释的搜索结果的需求越来越大，以便他们理解为什么提供某些搜索结果。同时，政府机构要求IR系统不仅要提供高质量的结果，还要提供合理的理由，以提高系统的可信度。该项目侧重于开发算法和框架，以提高现代红外系统的可核查性、可解释性和透明度。它将激发大规模的学术-产业合作，通过促进可靠和可解释的信息访问服务的发展，使数十亿用户受益。该项目将为可解析和可解释的IR开发通用和可重用框架。这个项目的研究将在两个方向上进行。第一个方向是建立新的模型内禀解释检索模型。这包括为检索操作开发透明的推理过程和决策边界，支持使用用户反馈进行结果探索的可解析功能，以及区分模型输入贡献的可跟踪信息流。第二个方向旨在为模型不可知的解释建立分析和模拟框架。这包括具有外部知识的事后解释系统，以及具有可解释输出的黑盒检索模型的模拟框架。除了模型内在和模型不可知论方法外，该项目还将研究众包任务和系统指标，以比较内在和不可知论解释的有效性。研究成果将包括多个公共基准数据集和可解释IR的评估平台，这将有助于研究界进行可持续和可重复的未来研究。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Information Retrieval (IR) systems are important for people for information access. For example, intelligent search engines are widely used in Web-based services such as web search, product search, and job search. Recently, sophisticated data and complicated black-box models have made modern IR systems less transparent to users. However, as more and more people rely on IR systems to guide their daily life and decision making, there has been growing needs of explainable search results, both for technical communities and the general public, so that they understand why certain search results are provided. Meanwhile, governmental agencies are demanding IR systems to provide not only high-quality results, but also reasonable justifications, so as to enhance the trustworthiness of the systems. This project focuses on developing algorithms and frameworks to improve the scrutability, explainability, and transparency of modern IR systems. It will inspire large-scale academic-industry collaboration, which benefits billions of users by facilitating the development of reliable and explainable information access services. This project will develop general and reusable frameworks for scrutable and explainable IR. Research in this project will be performed on two directions. The first direction aims at new retrieval models for model-intrinsic explanation. This includes developing transparent inference process and decision boundaries for retrieval actions, scrutable functions that support result exploration with user feedback, and traceable information flow to distinguish the contribution of model inputs. The second direction aims at building analytical and simulative framework for model-agnostic explanation. This includes post-hoc explanation systems with external knowledge, and a simulation framework over black-box retrieval models with explainable outputs. Besides model-intrinsic and model-agnostic approaches, this project will also investigate crowd-sourcing tasks and systematic metrics to compare the effectiveness of intrinsic and agnostic explanations. The research outcomes will include multiple public benchmark datasets and evaluation platforms for explainable IR, which will contribute to the research community for sustainable and reproducible future studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.