User behavior models for information access evaluation

信息访问评估的用户行为模型

• 批准号: 468812-2014
• 负责人: Smucker, Mark
• 金额: $ 4.7万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2017
• 资助国家: 加拿大
• 起止时间: 2017-01-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=618798
• 关键词: User; behavior; models; information; access

To improve search engines, and other information access systems, their performance must be measured in human terms. A key measure is the time taken to find required information and to complete the task at hand. Whether a user is searching for a single fact, or is researching a complex problem, the user wants to satisfy their information as fast as possible. Both researchers and engineers need a means to quickly and affordably estimate the rate at which users can satisfy their information needs with a given search engine. To produce accurate estimates of user performance, effectiveness measures must be designed that can adjust to different user tasks and computing devices. For example, users searching for a simple fact are expected to behave differently than users researching a complex problem. Likewise, the device used to access information can have a significant impact on how a user behaves. For mobile search, device constraints such as a small screen and a virtual keyboard need to be taken into consideration. Traditional effectiveness measures are ill-suited to estimate user effectiveness across different user tasks and access devices given their lack of user behavior models, which prevents adapting them to the search context. This work proposes to further develop the time-biased gain effectiveness measure, which can be adapted to the search context.The research undertaken by this project builds on existing work by Professors Smucker and Clarke. This existing work proposes and develops time-biased gain (TBG) effectiveness measures for evaluating search engine performance. Unlike traditional measures of search engine performance, TBG makes time a key part of the measure. Computations of TBG are calibrated to times and decision processes extracted from usage logs and user studies. As changes are made to a search engine's ranking algorithms, interface, and other components, TBG may be used to predict the impact of these changes on user performance. In order to advance this work to more complex scenarios and domains, the research will incorporate insights uncovered by Professor Cormack into relevance prediction. NSERC funding under this project serves to extend an existing Google Focus Award on context-aware mobile social networking. While developing technologies under that proposal, a need for improved evaluation methodologies became apparent. In addition to supporting the needs of this Google Focus Award, this project advances our general understanding of information access evaluation.

为了改进搜索引擎和其他信息访问系统，必须以人为本来衡量它们的性能。一个关键的衡量标准是找到所需信息和完成手头任务所需的时间。无论用户是在搜索单个事实，还是在研究复杂的问题，用户都希望尽可能快地满足他们的信息。研究人员和工程师都需要一种方法来快速和负担得起的估计率，用户可以满足他们的信息需求与给定的搜索引擎。为了准确估计用户性能，必须设计有效性度量，以适应不同的用户任务和计算设备。例如，搜索简单事实的用户与研究复杂问题的用户的行为预期不同。同样，用于访问信息的设备可能会对用户的行为产生重大影响。对于移动的搜索，需要考虑诸如小屏幕和虚拟键盘之类的设备约束。传统的有效性措施是不适合估计用户的有效性在不同的用户任务和访问设备，因为他们缺乏用户行为模型，这阻止了他们适应搜索上下文。这项工作提出了进一步发展的时间偏置增益有效性措施，这可以适应搜索context.The研究由该项目进行的建立在现有的工作由Smucker教授和克拉克。现有的工作提出并开发了时间偏置增益（TBG）的有效性措施，用于评估搜索引擎的性能。与传统的搜索引擎性能衡量标准不同，TBG将时间作为衡量标准的关键部分。TBG的计算根据从使用日志和用户研究中提取的时间和决策过程进行校准。当搜索引擎的排名算法、界面和其他组件发生变化时，TBG可用于预测这些变化对用户性能的影响。为了将这项工作推进到更复杂的场景和领域，这项研究将把Cormack教授发现的见解纳入相关性预测。NSERC在该项目下的资金用于扩展现有的谷歌焦点奖上下文感知移动的社交网络。在根据该提议开发技术的同时，显然需要改进评价方法。除了支持这个谷歌焦点奖的需要，这个项目推进了我们对信息访问评估的一般理解。