User behavior models for information access evaluation

为了改进搜索引擎和其他信息获取系统，它们的表现必须用人的标准来衡量。一个关键的衡量标准是查找所需信息和完成手头任务所需的时间。无论用户是在搜索单个事实，还是在研究复杂的问题，用户都希望尽可能快地满足他们的信息。研究人员和工程师都需要一种方法来快速且负担得起地估计用户使用给定搜索引擎满足其信息需求的速度。为了准确估计用户性能，必须设计能够适应不同用户任务和计算设备的有效性衡量标准。例如，搜索简单事实的用户应该与研究复杂问题的用户表现不同。同样，用于访问信息的设备可能会对用户的行为产生重大影响。对于移动搜索，需要考虑小屏幕和虚拟键盘等设备限制。由于缺乏用户行为模型，传统的效率指标不适合评估不同用户任务和访问设备的用户效率，这阻碍了它们适应搜索环境。这项工作建议进一步发展时间偏置的增益有效性度量，该度量可以适应搜索环境。

To improve search engines, and other information access systems, their performance must be measured in human terms. A key measure is the time taken to find required information and to complete the task at hand. Whether a user is searching for a single fact, or is researching a complex problem, the user wants to satisfy their information as fast as possible. Both researchers and engineers need a means to quickly and affordably estimate the rate at which users can satisfy their information needs with a given search engine. To produce accurate estimates of user performance, effectiveness measures must be designed that can adjust to different user tasks and computing devices. For example, users searching for a simple fact are expected to behave differently than users researching a complex problem. Likewise, the device used to access information can have a significant impact on how a user behaves. For mobile search, device constraints such as a small screen and a virtual keyboard need to be taken into consideration. Traditional effectiveness measures are ill-suited to estimate user effectiveness across different user tasks and access devices given their lack of user behavior models, which prevents adapting them to the search context. This work proposes to further develop the time-biased gain effectiveness measure, which can be adapted to the search context.