Pairwise comparisons or constrained optimization? A usability evaluation of techniques for eliciting decision priorities
Pairwise comparisons or constrained optimization? A usability evaluation of techniques for eliciting decision priorities
复制标题
DOI：
10.1111/itor.12907
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Edward Abel;I. Galpin;N. Paton;J. Keane
中科院分区：
文献类型：
--
作者：
Edward Abel;I. Galpin;N. Paton;J. Keane
决策支持方法为表达和传达通知决策的优先级提供了符号。虽然大量的文献已经探讨了这些符号和方法的理论优点，但很少有工作研究它们在实践中的可用性，这对它们被用户广泛采用至关重要。在本文中，我们探讨了两个著名的偏好启发技术，成对比较和约束优化的可用性。该技术进行了探索，通过两个对比人群工作者实验，初步评估识别，即能够确定最合适的配方为一个给定的任务，和其他合成，即能够构建配方为一个给定的任务。这些任务是基于一个案例研究，涉及源选择，一个众所周知的问题，在数据集成领域。经验评估的结果表明，总体而言，成对比较的性能明显高于约束优化，但每种技术的可用性评估之间的差异可以忽略不计。此外，我们观察到，参与者表现更好的技术不一定是他们认为更有用的技术。
Decision support methodologies provide notations for expressing and communicating the priorities that inform a decision. Although a substantial literature has explored the theoretical merits of such notations and methodologies, much less work has investigated their usability in practice, which is of vital importance for their widespread adoption by users. In this paper, we explore the usability of two well-known preference elicitation techniques, pairwise comparisons and constrained optimization . The techniques were explored through two contrasting crowd worker experiments, a preliminary one evaluating recognition , that is, the ability to identify the most suitable formulation for a given task, and the other synthesis , that is, the ability to construct formulations for a given task. The tasks are based on a case study involving source selection, a well-known problem in the data integration domain. The results of the empirical evaluation show that, overall, pairwise comparisons resulted in significantly higher performance than constrained optimization , yet there is negligible difference between the usability appraisals for each technique. Furthermore, we observed that the technique that participants perform better with is not necessarily the one that they consider more usable.