Accurate User Modeling of Search and Decision Making Tasks for Improved Offline Evaluation of Information Retrieval

准确的搜索和决策任务用户建模，以改进信息检索的离线评估

• 批准号: RGPIN-2020-04665
• 负责人: Smucker, Mark
• 金额: $ 3.5万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=758150
• 关键词: Accurate; User; Modeling; Search; Decision

My research program's objective is to make the search for information safer and more effective. Information retrieval, commonly known as search, is prevalent today with the ubiquitous use of web search engines (e.g. Google) and organizations providing information access via search engines. Engineers and researchers use test collections and offline effectiveness measures to improve the quality of search results. Offline evaluation allows engineers to affordably and efficiently simulate the effect of their algorithmic changes and measure the extent to which these changes help or harm search users. My research program works to improve the predictive ability of offline effectiveness measures by more accurately modeling user behaviour. Near-term objectives include 1) creating new methods to efficiently construct test collections that support user-model-based effectiveness measures and 2) improved prediction accuracy for effectiveness measures by modeling user interaction beyond a single query and search results pair. In the next 5 years and beyond, my research program will move to emphasize research on the evaluation of search in terms of the task outcome for a user rather than simply the documents retrieved by the user. For example, many search users turn to web search to help them make decisions about health-related issues. Unfortunately, users can perceive documents containing incorrect information as relevant to their decision-making tasks. We have shown that search results biased towards incorrect information can significantly reduce searchers' decision accuracy. Other researchers have shown that major search engines are biased and users may only view correct answers 45-52% of the time for health related searches. When search engines lead people to incorrect decisions about their health care, not only can money be wasted on scam treatments, but people's health can be harmed. My research program focuses on measuring search engine effectiveness so that researchers and engineers have the correct measurements to guide their work. I have proposed a line of work to study decision-making with search engines and to predict the accuracy of the decisions people reach when they use search engines for decision support. Our initial domain of study will be health-related search. The TREC Decision Track, which I co-organize as a part of my research program, provides a venue for researchers to improve the quality of search engines and reduce people's exposure to medical misinformation and improve their decisions. This proposed research can lead to improvements for everyone who uses search engines by helping them avoid misinformation and make better decisions. The research program also provides training for students in high value areas such as data analytics, user modeling, machine learning, and large scale data processing.

我的研究计划的目标是使搜索信息更安全，更有效。随着网络搜索引擎（例如b谷歌）和通过搜索引擎提供信息访问的组织的普遍使用，信息检索（通常称为搜索）在今天非常流行。工程师和研究人员使用测试集合和离线有效性度量来提高搜索结果的质量。离线评估允许工程师经济有效地模拟他们的算法变化的影响，并衡量这些变化对搜索用户的帮助或伤害的程度。我的研究项目是通过更准确地建模用户行为来提高线下有效性度量的预测能力。近期目标包括：1)创建新的方法来有效地构建支持基于用户模型的有效性度量的测试集合；2)通过对超越单个查询和搜索结果对的用户交互建模来提高有效性度量的预测准确性。在接下来的5年及更长的时间里，我的研究计划将转向强调根据用户的任务结果来评估搜索的研究，而不仅仅是用户检索到的文档。例如，许多搜索用户转向网络搜索来帮助他们做出与健康有关的决定。不幸的是，用户可能会认为包含错误信息的文档与他们的决策任务相关。我们已经证明，搜索结果偏向于不正确的信息会显著降低搜索者的决策准确性。其他研究人员已经表明，主要的搜索引擎是有偏见的，用户可能只有45-52%的时间在健康相关的搜索中看到正确的答案。当搜索引擎导致人们对他们的医疗保健做出错误的决定时，不仅金钱会被浪费在骗局治疗上，而且人们的健康也会受到损害。我的研究计划侧重于测量搜索引擎的有效性，以便研究人员和工程师有正确的测量来指导他们的工作。我已经提出了一系列工作来研究使用搜索引擎的决策，并预测人们在使用搜索引擎进行决策支持时做出的决策的准确性。我们最初的研究领域是与健康相关的搜索。TREC决策跟踪，我作为我的研究计划的一部分共同组织，为研究人员提供了一个场所，以提高搜索引擎的质量，减少人们接触到医疗错误信息并改善他们的决策。这项提议的研究可以通过帮助每个使用搜索引擎的人避免错误信息并做出更好的决定，从而为他们带来改进。该研究项目还为学生提供高价值领域的培训，如数据分析、用户建模、机器学习和大规模数据处理。