Learning-based evaluation of visual analytic systems
Learning-based evaluation of visual analytic systems
复制标题
基于学习的视觉分析系统评估
DOI：
--
复制
发表时间：
2010
期刊：
影响因子：
--
通讯作者：
W. Ribarsky
中科院分区：
文献类型：
--
作者：
Remco Chang;Caroline Ziemkiewicz;Roman Pyzh;Joseph Kielman;W. Ribarsky
由于可视化使用固有的独特限制和机会，可视化评估仍然是一个难题。尽管已经提出了许多潜在有用的方法，但在评估可视化社区所认为的理想的开放式探索和复杂任务解决的价值方面仍然存在巨大差距。在本文中，我们提出了一种定量评估视觉分析（VA）系统的方法，该方法基于测量用户在将知识重新应用到不同问题或领域时所学到的知识。这种方法的动机是基于这样的观察：VA 系统用户的最终目标是获得数据集、任务或工具本身的知识和专业知识。我们提出了一个框架，用于描述和衡量基于这三种类型的知识的分析过程中的知识增益，并讨论了评估每种知识的注意事项。我们建议，通过仔细设计测试来检查参与者如何重新应用从 VA 系统中学到的知识，可以更直接地评估可视化的效用。
Evaluation in visualization remains a difficult problem because of the unique constraints and opportunities inherent to visualization use. While many potentially useful methodologies have been proposed, there remain significant gaps in assessing the value of the open-ended exploration and complex task-solving that the visualization community holds up as an ideal. In this paper, we propose a methodology to quantitatively evaluate a visual analytics (VA) system based on measuring what is learned by its users as the users reapply the knowledge to a different problem or domain. The motivation for this methodology is based on the observation that the ultimate goal of a user of a VA system is to gain knowledge of and expertise with the dataset, task, or tool itself. We propose a framework for describing and measuring knowledge gain in the analytical process based on these three types of knowledge and discuss considerations for evaluating each. We propose that through careful design of tests that examine how well participants can reapply knowledge learned from using a VA system, the utility of the visualization can be more directly assessed.