Collaborative Research: User-Centered Visual Analytics Evaluation

翻译后摘要：这项建议的重点是扩展和编纂的方法和技术的可视化分析的评估。 它主要是基于基准的视觉分析挑战2010年（通过IEEE视觉分析科学和技术？被称为VAST-专题讨论会竞赛/挑战）作为一个测试平台，以推广以用户为中心的评估，以评估结合了联合收割机分析推理、视觉表示、人机交互（HCI）、复杂算法和协作工具的交互系统的有效性。 目前，还没有标准的方法来衡量交互式视觉分析系统的整体性能。 视觉分析系统不太适合传统的评估方法。 例如，精确度和召回率等简单方法并不适用，研究人员很少能够充分访问历史数据集，因为分析问题的复杂性存在于更广泛的实时领域。 执行这一项目的方法包括三项活动：1）收集数据集的各种整理与相关的地面真相/问题描述，2）开发自动度量和主观评估标准，以评估判断，以及3）调查这些方法的有效性和实用性以及在其他动态领域（如商业，情报，医学，应急响应）中重复使用的经验教训，等

Abstract: This proposal focuses on extending and codifying the methods and techniques for the evaluation of visual analytics. It is primarily based on benchmarking the Visual Analytics Challenge 2010 (organized through the IEEE Visual Analytics Science and Technology ? known as VAST- symposium contest/challenge) as a test bed to generalize user-centered evaluation to assess the effectiveness of interactive systems that combine analytical reasoning, visual representations, Human Computer Interactions (HCI), complex algorithms, and collaboration tools. At present, there are no standard methods for measuring the performance of interactive visual analytics systems as a whole. Visual analysis systems are not well suited for traditional evaluation methods. For example, simple methods such as precision and recall are not suitable and researchers rarely have had sufficient access to historical data sets with the complexity of the analytical problems that exist across wider, real-time domains. The means of approach to executing this project encompasses three activities: 1) gathering diverse collation of datasets with associated ground truth/problem descriptions, 2) developing automatic metrics and subjective evaluation criteria to assess judgment, and 3) the investigation of the effectiveness and utility of these methods and lessons learned for re-use in other dynamic fields such as business, intelligence, medicine, emergency response, etc.