CRII: III: A Bias-Aware Approach to Modeling Users in Interactive Information Retrieval

CRII：III：交互式信息检索中用户建模的偏差感知方法

• 批准号: 2106152
• 负责人: Jiqun Liu
• 金额: $ 17.5万
• 依托单位: University of Oklahoma Norman Campus
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-06-15 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106152&HistoricalAwards=false
• 关键词: CRII; III; Bias; Aware; Approach

People often act intuitively and are subject to systematic biases when making decisions under uncertainty due to their inability to calculate all the possible consequences of their choices - a fundamental cognitive phenomenon called "bounded rationality". Without proactive information supports, these decisions could be driven by misleading information, cognitive biases and heuristics and may result in significant deviations from desired outcomes: Health information seekers may easily trust medical misinformation that confirms their existing expectations. Students often heavily rely on top ranked results and stop at short satisficing answers, rather than exploring more credible and informative Web pages. Online shoppers tend to quickly accept immediate mediocre recommendations after encountering several bad quality products (with low reference levels), without examining all available options. By investigating users’ systematic biases, this project aims to break new grounds for information retrieval (IR) research and address fundamental bottlenecks in the development of bias-aware search systems. The outcomes of this project can help people better leverage the power of information through 1) incorporating the knowledge about their biases into search algorithms, and 2) proactively capturing bias-related search problems and promoting informed, unbiased decision-making.The project seeks to study users’ systematic biases and leverage the learned knowledge in improving the explanatory and predicative power of IR models. The technical aims of the project include: (1) understanding the relationships between search interactions and users’ systematic biases; (2) building bias-aware prediction models of search interactions; (3) developing a scalable and potentially transformative approach to modeling users and their decision-making processes under biases in interactive IR. To achieve these goals, the investigator will conduct a series of user studies and experiments. First, the research team will carry out controlled lab studies to examine the associations between users’ search interactions and several major systematic biases that have been widely confirmed by behavioral experiments, including reference dependence, framing effect, and loss aversion. Then, the team will extract new features and create bias-aware models for predicting users’ search behavior, experience, and problems. Finally, this project will apply deep neural networks in developing more fine-grained bias-aware models based on large scale test collections and search logs, and evaluate the performance of modified models in a wider range of search scenarios. The proposed models can provide a more solid behavioral and psychological basis for supporting the simulations of search interactions. Such simulations, properly constructed, could address major challenges in the design of boundedly-rational formal models and bias-aware intelligent systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人们经常凭直觉行事，在不确定的情况下做出决策时会受到系统性偏差的影响，因为他们无法计算出自己选择的所有可能后果--这是一种被称为“有限理性”的基本认知现象。如果没有主动的信息支持，这些决定可能会受到误导性信息、认知偏差和启发式的驱动，并可能导致与预期结果的重大偏离：健康信息寻求者可能很容易相信证实了他们现有期望的医疗错误信息。学生往往严重依赖排名靠前的结果，停留在令人满意的简短答案上，而不是探索更可信、更有信息量的网页。网上购物者在遇到几个质量不佳的产品(参考水平较低)后，往往会立即接受平庸的推荐，而不会检查所有可用的选择。通过调查用户的系统偏见，该项目旨在为信息检索(IR)研究开辟新的领域，并解决意识到偏见的搜索系统发展的根本瓶颈。这个项目的结果可以帮助人们更好地利用信息的力量，通过1)将关于他们的偏见的知识纳入搜索算法，以及2)主动捕获与偏见相关的搜索问题，并促进知情的、公正的决策。该项目旨在研究用户的系统性偏见，并利用所学的知识来改进IR模型的解释和预测能力。该项目的技术目标包括：(1)了解搜索交互和用户系统偏差之间的关系；(2)建立搜索交互的偏见感知预测模型；(3)开发一种可扩展的、具有潜在变革性的方法来模拟交互式IR中的用户及其决策过程。为了实现这些目标，调查员将进行一系列用户研究和实验。首先，研究小组将进行受控实验室研究，以检查用户的搜索交互与几个主要的系统性偏差之间的关联，这些偏差已被行为实验广泛证实，包括参考依赖、框架效应和损失厌恶。然后，该团队将提取新的功能并创建偏见感知模型，以预测用户的搜索行为、体验和问题。最后，该项目将应用深度神经网络来开发基于大规模测试集合和搜索日志的更细粒度的偏见感知模型，并评估改进后的模型在更广泛的搜索场景中的性能。所提出的模型可以为搜索交互的模拟提供更坚实的行为和心理基础。如果构建得当，这种模拟可以解决有限理性正式模型和偏见感知智能系统设计中的主要挑战。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Toward A Two-Sided Fairness Framework in Search and Recommendation

• DOI: 10.1145/3576840.3578332
• 发表时间: 2023-03
• 期刊: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval
• 作者: Jiqun Liu

A Reference Dependence Approach to Enhancing Early Prediction of Session Behavior and Satisfaction

• DOI: 10.1145/3529372.3533294
• 发表时间: 2022-06
• 期刊: 2022 ACM/IEEE Joint Conference on Digital Libraries (JCDL)
• 作者: T. Brown;Jiqun Liu

Investigating the role of in-situ user expectations in Web search

• DOI: 10.1016/j.ipm.2023.103300
• 发表时间: 2023-05
• 期刊: Inf. Process. Manag.
• 作者: Ben Wang;Jiqun Liu

Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation

• DOI: 10.1016/j.ipm.2022.103007
• 发表时间: 2022-09
• 期刊: Inf. Process. Manag.
• 作者: Jiqun Liu

Investigating the Relationship between In‐Situ User Expectations and Web Search Behavior

• DOI: 10.1002/pra2.740
• 发表时间: 2022-10
• 期刊: Proceedings of the Association for Information Science and Technology
• 作者: Ben Wang;Jiqun Liu