CGV: Small: Toward Objective, In-Situ, and Generalizable Evaluation of Visual Analytics by Integrating Brain Imaging with Cognitive Factors Analysis

CGV：小：通过将脑成像与认知因素分析相结合，实现视觉分析的客观、原位和可推广的评估

• 批准号: 1218170
• 负责人: Remco Chang
• 金额: $ 49.99万
• 依托单位: Tufts University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1218170&HistoricalAwards=false
• 关键词: CGV; Small; Toward; Objective; Situ

Evaluating complex, interactive visual analytics systems is challenging for many reasons. The exploratory nature of using visualization makes quantitative measurements of individual components of the visualization task infeasible. The range of potential users and their differing goals render standardized metrics too restrictive. And environmental conditions can influence experimental outcomes, making it difficult to compare and generalize the results of separate evaluations. Although numerous methods have been proposed for evaluating visualizations and HCI, few can be readily applied to objectively evaluate visual analytics systems in real-world settings. In this project the PI will address this open challenge by developing a method to evaluate complex, interactive visual analytics systems using objective measures that can be performed in-situ to yield reproducible, generalizable results. His approach is to integrate noninvasive brain imaging using functional near-infrared spectroscopy (fNIRS) and cognitive factors measurements. The PI argues this will allow him to address issues in visual analytics evaluation by explaining the user's cognitive processes at a deeper level. Lightweight, noninvasive brain imaging techniques such as fNIRS have become more mature and reliable in recent years. fNIRS is easy to set up, robust to movement, and has been demonstrated in studies to be effective in determining a user's cognitive load, preferences, and perception when using a visualization. Cognitive factors such as locus of control, spatial visualization ability, and perceptual speed have recently been shown to correlate with a user's ability to interact with a visualization, and can be generalized to predict the behavioral patterns of users with different cognitive profiles. Both approaches aim to better understand the user's cognitive state and abilities. In this research cognitive factors measurements will provide low-level, baseline information about the user which is stable and unchanging ("traits"), while fNIRS provides immediate, real-time feedback on the user's current cognitive state when interacting with a visualization ("states"). In practice, without accounting for the individual user's traits, it is difficult to generalize the signals provided by fNIRS (for example, right- and left-handed subjects produce very different brain signals). By combining information on both "traits" and "states," evaluation results can generalize to a larger population based on cognitive profiles. The PI and his team have conducted two preliminary experiments that demonstrate the feasibility of the approach. They have replicated the classic experiment by Cleveland and McGill using fNIRS, and successfully distinguished participants' brain signals when using bar charts versus pie charts. In another experiment, they successfully correlated participants' locus of control with their ability to use hierarchical visualizations with different visual metaphors.Broader Impacts: The results of this work will have both immediate and long term impact on the field of visualization. In the short term, project outcomes will provide a robust and reliable evaluation mechanism for measuring the effectiveness of complex, interactive visual analytics systems. Ultimately, the findings of this research will help open the human-cognition "black box" and illuminate how interacting with a visualization helps a user gain insight. Such an understanding may in turn lead to the realization of insight-based evaluation and the emergence of a new visualization theory that is based on human cognitive processing. Finally, by integrating real-time fNIRS into visual analytics systems, effective adaptive mixed-initiative visual analytics systems can become one step closer to reality.

评估复杂的交互式可视化分析系统具有挑战性，原因有很多。 使用可视化的探索性质使得可视化任务的各个组成部分的定量测量是不可行的。 潜在用户的范围和他们不同的目标使得标准化的度量标准限制性太强。 环境条件会影响实验结果，因此很难比较和概括单独评估的结果。 尽管已经提出了许多方法来评估可视化和HCI，但很少有方法可以在现实世界中客观地评估可视化分析系统。 在这个项目中，PI将通过开发一种方法来解决这一开放性挑战，该方法使用可以在现场进行的客观测量来评估复杂的交互式视觉分析系统，以产生可重现的，可推广的结果。 他的方法是使用功能性近红外光谱（fNIRS）和认知因素测量来整合非侵入性脑成像。 PI认为，这将使他能够通过在更深层次上解释用户的认知过程来解决视觉分析评估中的问题。 近年来，fNIRS等轻便、无创的脑成像技术已变得更加成熟和可靠。 fNIRS易于设置，对运动具有鲁棒性，并且在研究中已被证明在使用可视化时可有效确定用户的认知负荷、偏好和感知。 认知因素，如控制点，空间可视化能力和感知速度最近已被证明与用户的能力，以互动的可视化，并可以概括为预测用户的行为模式与不同的认知配置文件。 这两种方法都旨在更好地了解用户的认知状态和能力。 在这项研究中，认知因素测量将提供有关用户的低水平基线信息，这些信息是稳定和不变的（“特质”），而fNIRS在与可视化交互时提供有关用户当前认知状态的即时实时反馈（“状态”）。 在实践中，如果不考虑个人用户的特征，很难概括fNIRS提供的信号（例如，右手和左手受试者产生非常不同的大脑信号）。 通过结合关于“特质”和“状态”的信息，评估结果可以根据认知概况推广到更大的人群。 PI和他的团队已经进行了两个初步实验，证明了这种方法的可行性。 他们使用fNIRS复制了克利夫兰和麦吉尔的经典实验，并成功区分了使用条形图和饼图时参与者的大脑信号。 在另一项实验中，他们成功地将参与者的控制点与他们使用不同视觉隐喻的层次可视化的能力联系起来。 这项工作的结果将对可视化领域产生直接和长期的影响。 在短期内，项目成果将为衡量复杂的交互式可视化分析系统的有效性提供一个强大而可靠的评估机制。 最终，这项研究的结果将有助于打开人类认知的“黑匣子”，并阐明与可视化交互如何帮助用户获得洞察力。 这种理解反过来又可能导致基于洞察力的评价的实现和基于人类认知过程的新的可视化理论的出现。 最后，通过将实时fNIRS集成到视觉分析系统中，有效的自适应混合主动视觉分析系统可以更接近现实。