CRII: III: A Bias-Aware Approach to Modeling Users in Interactive Information Retrieval

由于人们无法计算自己选择的所有可能后果，在不确定的情况下做出决定时，人们往往凭直觉行事，并受到系统性偏见的影响——这是一种被称为“有限理性”的基本认知现象。如果没有前瞻性的信息支持，这些决定可能受到误导性信息、认知偏差和启发式的驱动，并可能导致与预期结果的重大偏差：健康信息寻求者可能很容易相信证实其现有期望的医学错误信息。学生往往严重依赖排名靠前的结果，止步于简短的满意答案，而不是探索更可信、信息更丰富的网页。在线购物者在遇到一些质量差的产品（参考水平低）后，往往会很快接受平庸的推荐，而不会检查所有可用的选择。通过调查用户的系统性偏见，该项目旨在为信息检索（IR）研究开辟新的领域，并解决偏见感知搜索系统发展中的基本瓶颈。这个项目的成果可以帮助人们更好地利用信息的力量：1)将关于他们偏见的知识纳入搜索算法；2)主动捕捉与偏见相关的搜索问题，促进知情、公正的决策。该项目旨在研究用户的系统性偏见，并利用所学知识提高IR模型的解释和预测能力。该项目的技术目标包括：(1)理解搜索交互与用户系统性偏见之间的关系；(2)建立搜索交互的偏见感知预测模型；(3)开发一种可扩展且具有潜在变革性的方法，对交互式IR中存在偏见的用户及其决策过程进行建模。为了实现这些目标，研究者将进行一系列的用户研究和实验。首先，研究团队将进行控制实验室研究，以检查用户搜索交互与几个主要系统偏差之间的关联，这些偏差已被行为实验广泛证实，包括参考依赖、框架效应和损失厌恶。然后，团队将提取新功能并创建偏见感知模型，以预测用户的搜索行为、体验和问题。最后，本项目将基于大规模测试集合和搜索日志，应用深度神经网络开发更细粒度的偏差感知模型，并在更广泛的搜索场景中评估修改后的模型的性能。所提出的模型可以为支持搜索交互的模拟提供更坚实的行为和心理基础。这样的模拟，如果构造得当，可以解决有界理性形式模型和偏见感知智能系统设计中的主要挑战。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

People often act intuitively and are subject to systematic biases when making decisions under uncertainty due to their inability to calculate all the possible consequences of their choices - a fundamental cognitive phenomenon called "bounded rationality". Without proactive information supports, these decisions could be driven by misleading information, cognitive biases and heuristics and may result in significant deviations from desired outcomes: Health information seekers may easily trust medical misinformation that confirms their existing expectations. Students often heavily rely on top ranked results and stop at short satisficing answers, rather than exploring more credible and informative Web pages. Online shoppers tend to quickly accept immediate mediocre recommendations after encountering several bad quality products (with low reference levels), without examining all available options. By investigating users’ systematic biases, this project aims to break new grounds for information retrieval (IR) research and address fundamental bottlenecks in the development of bias-aware search systems. The outcomes of this project can help people better leverage the power of information through 1) incorporating the knowledge about their biases into search algorithms, and 2) proactively capturing bias-related search problems and promoting informed, unbiased decision-making.The project seeks to study users’ systematic biases and leverage the learned knowledge in improving the explanatory and predicative power of IR models. The technical aims of the project include: (1) understanding the relationships between search interactions and users’ systematic biases; (2) building bias-aware prediction models of search interactions; (3) developing a scalable and potentially transformative approach to modeling users and their decision-making processes under biases in interactive IR. To achieve these goals, the investigator will conduct a series of user studies and experiments. First, the research team will carry out controlled lab studies to examine the associations between users’ search interactions and several major systematic biases that have been widely confirmed by behavioral experiments, including reference dependence, framing effect, and loss aversion. Then, the team will extract new features and create bias-aware models for predicting users’ search behavior, experience, and problems. Finally, this project will apply deep neural networks in developing more fine-grained bias-aware models based on large scale test collections and search logs, and evaluate the performance of modified models in a wider range of search scenarios. The proposed models can provide a more solid behavioral and psychological basis for supporting the simulations of search interactions. Such simulations, properly constructed, could address major challenges in the design of boundedly-rational formal models and bias-aware intelligent systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.