Cloud-based multi-modal visualization technique for emotional response in human-computer interaction

概述*这项提议的目的是调查基于云的用户与计算机软件交互的情感状态的可视化技术的商业潜力。其新颖性在于对多个数据源的三角测量和机器学习技术的使用。该技术可以帮助可用性专家通过识别例如界面的哪些部分产生正面和负面情绪来对用户体验和界面的人体工学特征进行更全面的诊断。正在研究中的可用性测试技术的好处通过一个原型系统进行了演示，并在网站和移动应用程序评估等各种背景下发表的科学论文中进行了描述。目前PhotoBoth的原型已经被加拿大多个行业的公司成功地用于合作研究环境。**传统的凝视热图被用于眼睛跟踪，作为聚合凝视数据的直观表示。它们的主要用途是帮助回答这样一个问题：“在界面中，人们倾向于看哪里？”在我们的可视化方法中，用户的凝视现在作为一种手段，将情感信号映射到用户界面上。生成的热图表示界面上的情绪，并有助于回答以下问题：“在界面中，人们倾向于在哪里做出情绪反应？”*

Summary****The objective of this proposal is to investigate the commercial potential of a cloud-based visualisation technique of the emotional state of users interacting with computer software. The novelty lies in the triangulation of multiple data sources and the utilization of machine learning techniques. This technique can help usability experts make a more comprehensive diagnostic of the user experience and of the ergonomic characteristics of an interface by identifying, for example, which parts of an interface generate positive and negative emotions. The benefits of the usability testing technique under investigation were demonstrated with a prototype system and were described in published scientific papers in various contexts, such as websites and mobile applications evaluation. The current prototype of Photobooth has been successfully used in a collaborative research context by Canadian companies across a number of industries.**Traditional gaze heatmaps are used in eye tracking as intuitive representations of aggregated gaze data. Their main use is to help answer the question: "Where in the interface do people tend to look?" In our visualization method, the users' gaze now serves as a means of mapping emotional signals onto the user interface. The resulting heatmaps represent emotions on the interface, and can help answer the following question: "Where in the interface do people tend to react emotionally?" **************