Balancing Relevance Criteria through Multi-Objective Optimization
Balancing Relevance Criteria through Multi-Objective Optimization
复制标题
通过多目标优化平衡相关性标准
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
M. de Rijke
中科院分区：
文献类型：
--
作者：
J. Doorn;Daan Odijk;D. Roijers;M. de Rijke
信息检索系统的离线评估通常集中在一个单一的有效性措施，模型的效用为一个典型的用户。这种衡量标准通常结合了基于行为的排名折扣和文档效用的概念，后者抓住了话题性的单一相关性标准。然而，对于个人用户的相关性标准，如可信度，声誉或可读性可以强烈影响的效用。此外，对于不同的信息需求，效用可以是这些标准的不同混合。由于关注于单一度量，IR系统的离线优化在平衡相关性准则时不考虑不同偏好。我们建议通过将多个相关性标准视为目标并学习一组提供不同权衡的排名器来缓解这一问题。这些目标。我们在基于收益的评估框架内将文档效用建模为相关性标准的加权组合。使用学习的集合，我们能够根据排名的值和偏好w.r.t.相关性标准。在一个数据集上注释的可读性和一个网络搜索数据集注释的子主题的相关性，我们演示了如何权衡之间可以明确。我们表明，有不同的相关性标准之间的权衡。
Offline evaluation of information retrieval systems typically focuses on a single effectiveness measure that models the utility for a typical user. Such a measure usually combines a behavior-based rank discount with a notion of document utility that captures the single relevance criterion of topicality. However, for individual users relevance criteria such as credibility, reputability or readability can strongly impact the utility. Also, for different information needs the utility can be a different mixture of these criteria. Because of the focus on single metrics, offline optimization of IR systems does not account for different preferences in balancing relevance criteria. We propose to mitigate this by viewing multiple relevance criteria as objectives and learning a set of rankers that provide different trade-offs w.r.t. these objectives. We model document utility within a gain-based evaluation framework as a weighted combination of relevance criteria. Using the learned set, we are able to make an informed decision based on the values of the rankers and a preference w.r.t. the relevance criteria. On a dataset annotated for readability and a web search dataset annotated for sub-topic relevance we demonstrate how trade-offs between can be made explicit. We show that there are different available trade-offs between relevance criteria.