Cloud-based multi-modal visualization technique for emotional response in human-computer interaction

摘要****本提案的目的是研究基于云的用户与计算机软件交互的情感状态可视化技术的商业潜力。其新颖之处在于对多个数据源的三角测量和机器学习技术的利用。这种技术可以帮助可用性专家对用户体验和界面的人体工程学特征做出更全面的诊断，例如，通过识别界面的哪些部分会产生积极和消极的情绪。研究中的可用性测试技术的好处通过一个原型系统进行了演示，并在各种环境（如网站和移动应用程序评估）中发表了科学论文。目前Photobooth的原型已经成功地应用于加拿大多个行业的公司的合作研究中。**传统的注视热图用于眼动追踪，作为聚合注视数据的直观表示。它们的主要用途是帮助回答这样一个问题：“人们倾向于在界面的哪里寻找？”在我们的可视化方法中，用户的凝视现在作为一种将情感信号映射到用户界面上的手段。由此产生的热图代表了界面上的情感，可以帮助回答以下问题：“在界面的什么地方，人们倾向于做出情感反应？”**************

Summary****The objective of this proposal is to investigate the commercial potential of a cloud-based visualisation technique of the emotional state of users interacting with computer software. The novelty lies in the triangulation of multiple data sources and the utilization of machine learning techniques. This technique can help usability experts make a more comprehensive diagnostic of the user experience and of the ergonomic characteristics of an interface by identifying, for example, which parts of an interface generate positive and negative emotions. The benefits of the usability testing technique under investigation were demonstrated with a prototype system and were described in published scientific papers in various contexts, such as websites and mobile applications evaluation. The current prototype of Photobooth has been successfully used in a collaborative research context by Canadian companies across a number of industries.**Traditional gaze heatmaps are used in eye tracking as intuitive representations of aggregated gaze data. Their main use is to help answer the question: "Where in the interface do people tend to look?" In our visualization method, the users' gaze now serves as a means of mapping emotional signals onto the user interface. The resulting heatmaps represent emotions on the interface, and can help answer the following question: "Where in the interface do people tend to react emotionally?" **************