III: Small: Characterizing and Evaluating Whole Session Interactive Information Retrieval

这项研究解决了当代生活中的一个新的重要问题。随着人们越来越习惯于使用Web查找信息，他们越来越多地使用Web来解决越来越复杂和对个人非常重要的信息问题。然而，目前的Web搜索引擎已经开发并专门调整为帮助人们找到简单的、主要是事实的信息，通常是作为对单个简单查询的单一响应列表。但当他们试图解决新类型的问题时，人们需要参与比一个查询-一个回答范式假设的更长的信息寻找插曲。他们可能还需要从事许多活动，而不仅仅是点击搜索结果，如阅读、评估、比较和使用信息。目前的网络搜索引擎还不能充分支持这种信息搜索和使用模式。这项研究通过研究人们为什么参与如此复杂的信息寻找(即，促使他们这样做的原因)，以及他们在信息寻求过程中试图实现的目标(他们的搜索意图)来解决这个问题。这项研究的最终目标是设计和评估新型搜索引擎，以支持人们实现导致他们从事信息搜索的目标。从本质上讲，这意味着能够个性化系统对个人的支持，以及个人的目标和背景。具体地说，这项研究将建立人们在信息寻求情节中的行为、导致他们从事信息寻求的动机目标以及他们在信息寻求情节中任何时候的具体意图之间的关系。这将使开发的系统能够预测如何最好地支持个人解决他们的信息问题。例如，这个项目的发现可以帮助建立一个系统，自动识别搜索者正在购买汽车，并帮助他/她比较新车和二手车、购买和租赁的成本效益，最终做出明智的决定。研究将通过开发补充电子学校课程和图书/信息科学方案等课程的模块与教育活动相结合。这一点很重要，因为广泛的学生将学习新的搜索方法以及相关的用户研究和评估。关于可观察到的搜索者行为与搜索者希望在信息寻找插曲期间的任何特定点实现的较高级别意图之间的关系知之甚少，也没有好的方法或措施来评估交互式信息检索系统对整个信息搜索插曲评估的支持，或者用于评估在信息搜索插曲期间对任何特定搜索者意图的支持。该项目涉及以下问题：通过观察搜索者的行为，识别搜索者在信息搜索插曲过程中的意图；制定评价交互式信息检索系统在信息搜索插曲过程中支持这些意图的方法和措施；以及制定评价交互式信息检索系统支持整个信息搜索插曲的性能的措施和方法。所观察到的行为包括例如眼睛注视、鼠标移动、点击、跟随链接、页面转换和查询重构。该项目分三个阶段进行。首先，它建立并扩展了前人和当前关于将可观察到的搜索者行为与高级搜索者意图联系起来的研究结果，目的是根据这些行为及其关联的意图来表征和分割信息寻找情节，并开发信息寻求情节行为的模型。这是用以前收集的数据和从新的用户研究中得出的数据进行的。第二阶段通过浏览器插件记录工具和对参与者的采访，收集关于寻找剧集的“真实”信息的数据。分析这些数据是为了发现新的激励任务，识别新的信息寻求事件意图，并加强信息寻求事件行为的模型。第三阶段，包括对执行复杂信息搜索任务的人的进一步研究，确定在信息搜索过程中和整个信息搜索过程中意图的目标，并制定和测试支持这些目标的措施，以及收集应用这些措施所需数据的方法。第三阶段还研究了行为模型在信息寻求事件模拟中的应用。本质上，这个项目将产生预测搜索者意图的统计模型和算法，从而产生更好的个性化和推荐方法。这些结果将是新颖而有意义的，因为它们将允许我们通过用户的信息搜索片段来解决目标/任务完成问题，而不仅仅是搜索改进(S)。该项目的成果将通过项目网站(http://inforetrieval.org/iir/).)传播该网站将具体包括技术报告、出版物以及指向该项目产生的数据集和开放源码软件的链接。

This research addresses a newly important issue in contemporary life. As people become more accustomed to using the Web for finding information, they are increasingly using it for addressing ever more complex and personally important information problems. However, current Web search engines have been developed and specifically tuned to helping people find simple, mostly factual information, usually as a single response list to a single, simple query. But when they try to address the new types of problems, people need to engage in longer information seeking episodes than the one query-one response paradigm assumes. They may also need to engage in many activities other than just clicking on a search result, such as reading, evaluating, comparing and using information. Current Web search engines do not sufficiently support this model of information seeking and use. This research addresses this problem by studying why people engage in such complex information seeking (that is, the reasons that motivate them to do this), and what they try to accomplish during the course of an information seeking episode (their search intentions). The end-goal of this research is to design and evaluate new types of search engines for supporting people in accomplishing the goals that have led them to engage in information seeking. This means, in essence, being able to personalize system support to the individual, and the individual's goals and context. Specifically, this research will establish relationships between people's behaviors during an information seeking episode, the motivating goals that led them to engage in information seeking, and their specific intentions at any point in an information seeking episode. This will enable development of systems that will be able to predict how best to support the individual person in addressing their information problem. For example, the findings from this project could help build a system that automatically identifies that a searcher is shopping for a car, and help him/her compare cost-benefits of new vs. used cars, buying vs. leasing, and eventually making an informed decision. Research will be integrated with educational activities via developing modules to supplement courses in iSchools and library/information science programs, etc. This is important, since a broad range of students would learn about new methods of searching and related user studies and evaluation. Little is known about the relationships between observable searcher behaviors and the higher-level intentions which a searcher wishes to accomplish at any particular point during an information seeking episode, nor are there good methods or measures for evaluating interactive information retrieval system support for whole information seeking episode evaluation, or for evaluation of support for any particular searcher intention during an information seeking episode. This project addresses the problems of: recognizing searcher intentions during the course of an information seeking episode through observation of searcher behavior; developing methods and measures for evaluation of interactive information retrieval system support for those intentions during the course of an information seeking episode; and, developing measures and methods for evaluating the performance of the interactive information retrieval system in support of the entire information seeking episode. The observed behaviors include, e.g., eye-fixations, mouse movements, clicking, following links, page transitions and query reformulations. The project proceeds in three stages. First, it builds on and extends the results of previous and current research in relating observable searcher behaviors to high-level searcher intentions, with the goal of characterizing and segmenting information seeking episodes according to these behaviors and their associated intentions, and to develop models of information seeking episode behaviors. This is conducted both with previously collected data, and on data derived from a new user studies. Stage 2 collects data on "real" information seeking episodes, through a browser plug-in logging tool, and interviews with participants. These data are analyzed to discover new motivating tasks, to identify new information seeking episode intentions, and to enhance the models of information seeking episode behaviors. Stage 3, including a further study of people conducting complex information seeking tasks, identifies the goals of intentions during the information seeking episode and for the whole information seeking episode, and develops and tests measures of support for those goals, and methods for gathering the data required for applying the measures. Stage 3 also investigates the application of the behavior models for simulation of information seeking episodes. In essence, this project will result in statistical models and algorithms that predict a searcher's intentions, leading to better ways for personalization and recommendations. These outcomes will be novel and significant because they will allow us to address goal/task accomplishment, and not just search improvement, through a user's information seeking episode(s). The results from this project will be disseminated through the project website (http://inforetrieval.org/iir/). The website will specifically include technical reports, publications, and links to datasets and open-source software resulted from this project.