The Effects of Relevance Feedback Quality and Quantity in Interactive Relevance Feedback: A Simulation Based on User Modeling
The Effects of Relevance Feedback Quality and Quantity in Interactive Relevance Feedback: A Simulation Based on User Modeling
复制标题
交互式相关反馈中相关反馈质量和数量的影响：基于用户建模的模拟
DOI：
--
复制
发表时间：
2006
期刊：
影响因子：
--
通讯作者：
Ari Pirkola
中科院分区：
文献类型：
--
作者：
Heikki Keskustalo;K. Järvelin;Ari Pirkola
与真实用户进行相关性反馈有效性的实验既耗时又昂贵。这使得快速测试的模拟成为可取的。我们定义了一个用户模型，它有助于量化模拟相关性反馈中涉及的一些交互决策。首先，相关性标准定义了用户接受与其需求相关的文档的相关性阈值。其次，浏览努力是指用户浏览初始检索文档列表以提供反馈的耐心。第三，反馈努力是指用户收集反馈文档的努力和能力。我们使用该模型在实验室环境中构建了几个模拟相关反馈场景。使用提供分级相关性评估的 TREC 数据，我们研究了反馈文档的质量和数量对相关性反馈有效性的影响，并将其与伪相关性反馈进行比较。我们的结果表明，人们可以通过少量高度相关的反馈来补偿大量相关但低质量的反馈。
Experiments on the effectiveness of relevance feedback with real users are time-consuming and expensive. This makes simulation for rapid testing desirable. We define a user model, which helps to quantify some interaction decisions involved in simulated relevance feedback. First, the relevance criterion defines the relevance threshold of the user to accept documents as relevant to his/her needs. Second, the browsing effort refers to the patience of the user to browse through the initial list of retrieved documents in order to give feedback. Third, the feedback effort refers to the effort and ability of the user to collect feedback documents. We use the model to construct several simulated relevance feedback scenarios in a laboratory setting. Using TREC data providing graded relevance assessments, we study the effect of the quality and quantity of the feedback documents on the effectiveness of the relevance feedback and compare this to the pseudo-relevance feedback. Our results indicate that one can compensate large amounts of relevant but low quality feedback by small amounts of highly relevant feedback.