Conceptual and Methodological Issues in Evaluating Multidimensional Visualizations for Decision Support
Conceptual and Methodological Issues in Evaluating Multidimensional Visualizations for Decision Support
复制标题
DOI：
10.1109/tvcg.2017.2745138
复制
发表时间：
2018-01-01
影响因子：
5.2
通讯作者：
Dragicevic, Pierre
中科院分区：
文献类型：
--
作者：
Dimara, Evanthia;Bezerianos, Anastasia;Dragicevic, Pierre
我们探索如何严格评估多维可视化支持决策的能力。我们首先定义多属性选择任务，这是一种通常通过此类可视化执行的决策任务。然后，我们确定哪些现有的多维可视化与此类任务兼容，并着手评估三种基本可视化：平行坐标、散点图矩阵和表格可视化。我们的方法包括首先为参与者提供低级分析任务，以确保他们正确理解可视化及其交互。然后，参与者将接受多属性选择任务，包括选择度假套餐。我们通过多个客观和主观指标来评估决策支持，包括基于所做选择和自我报告的属性偏好之间的一致性的决策准确性指标。我们发现这三种可视化在大多数指标上都具有可比性，但表格可视化略有优势。特别是，表格可视化可以让参与者更快地做出决策。因此，尽管决策时间通常不是评估决策支持的核心，但当可视化达到类似的决策准确性时，它可以用作决胜局。我们的结果还表明，评估选择置信度的间接方法可能比直接方法更好地区分可视化。最后，我们讨论了我们的方法的局限性和未来工作的方向，例如需要更敏感的决策支持指标。
We explore how to rigorously evaluate multidimensional visualizations for their ability to support decision making. We first define multi-attribute choice tasks, a type of decision task commonly performed with such visualizations. We then identify which of the existing multidimensional visualizations are compatible with such tasks, and set out to evaluate three elementary visualizations: parallel coordinates, scatterplot matrices and tabular visualizations. Our method consists in first giving participants low-level analytic tasks, in order to ensure that they properly understood the visualizations and their interactions. Participants are then given multi-attribute choice tasks consisting of choosing holiday packages. We assess decision support through multiple objective and subjective metrics, including a decision accuracy metric based on the consistency between the choice made and self-reported preferences for attributes. We found the three visualizations to be comparable on most metrics, with a slight advantage for tabular visualizations. In particular, tabular visualizations allow participants to reach decisions faster. Thus, although decision time is typically not central in assessing decision support, it can be used as a tie-breaker when visualizations achieve similar decision accuracy. Our results also suggest that indirect methods for assessing choice confidence may allow to better distinguish between visualizations than direct ones. We finally discuss the limitations of our methods and directions for future work, such as the need for more sensitive metrics of decision support.