CHS: Small: Enhancing EEG-based Emotion Estimation with Transfer Learning, Priming, and Virtual Reality

情绪影响着我们每天所做的决定。然而，情绪是很难衡量的。问卷调查是传统的情绪测量方法，需要花费大量的时间，并且会造成干扰。此外，人们并不总是知道自己的情绪——想想当一个人大喊大叫的时候，“我没有大喊大叫！”基于佩戴在身体或头部的传感器的直接情绪测量系统已经显示出前景，但还没有准备好与调查相竞争。该项目包括一些新技术，旨在提高这些系统的可靠性和性能。该项目将为可穿戴设备奠定基础，这些设备可以在实验室以外的环境中使用，如教室和主题公园。从那以后，这项研究可能会导致人机系统的发展，这种系统可以实时调整以保持一个人的兴趣。这样的系统对各级教育、娱乐和其他领域都很有价值。在项目期间，妇女和代表性不足的少数群体成员将作为研究人员和研究参与者参与。该项目是第一个将情绪估计脑机接口（bci）与三种方法结合起来的项目之一：迁移学习、情绪启动和虚拟现实。迁移学习有望帮助BCI系统检测真实的情绪，而不是设定特定的大脑反应。迁移学习技术将应用于来自不同情绪激发范式的数据，以研究标准方法的训练数据在新环境中的预期效果。我们将使用心理学上另一种叫做“情绪启动”的训练技术，并研究其对迁移学习的影响。情绪启动有望提高情绪激发的准确性，提高训练数据的质量，从而提高系统性能。虚拟现实将通过比图片和视频更大的范围来操纵参与者的唤醒状态，以产生更多可转移的结果。总之，这些研究有望显著提高这些系统的性能，更重要的是，提高这些系统的跨任务可靠性。由于情绪研究几乎影响到人类研究的每个领域，因此开发一种实时测量有效情绪的系统有可能推动跨学科的转变。总的来说，这些测试将填补科学文献中的知识空白，并告知如何训练未来的通用情感估计系统。此外，该项目将提供共享数据，这些数据可能会在未来的研究人员使用时在脑机接口领域取得更大的进步。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Emotions influence the decisions we make every day. However, emotions are difficult to measure. Surveys, the traditional method of emotion measurement, take significant time and cause interruptions. Further, people do not always know their own emotions -- think of the times when a person has shouted, "I'm not shouting!" Direct emotion-measuring systems, based on sensors worn on the body or head, have shown promise but are not yet ready to compete with surveys. This project includes several new techniques designed to increase the reliability and performance of these systems. The project will lay the groundwork for wearable devices that can be used outside the laboratory, in environments as varied as classrooms and theme parks. From there, this research could lead to the development of human-computer systems which adjust in real-time to maintain a person's interest. Such systems would be valuable for all levels of education, as well as entertainment and other fields. During the project, women and members of underrepresented minorities will participate, both as researchers and as research participants. This project is one of the first to combine emotion-estimating Brain-Computer Interfaces (BCIs) with three methods: transfer learning, emotion priming, and virtual reality. Transfer learning is expected to help BCI systems detect real emotions rather than setting-specific brain responses. Transfer learning techniques will be applied to data from different emotion elicitation paradigms to study how well training data from standard methods can be expected to work in new environments. An alternative training technique from psychology called "emotion priming" will be used, and the effects on transfer learning will be studied. Emotion priming is expected to increase the accuracy of emotion elicitation, increasing the quality of training data and thus system performance. Virtual reality will be used to manipulate the participants' arousal states through a larger range than is possible through pictures and videos, in order to generate more transferable results. Together, these investigations are expected to dramatically improve the performance and, critically, the cross-task reliability of these systems. Because the study of emotion impacts nearly every field of human study, the development of a system for real-time measurement of valid emotions has the potential to advance cross-disciplinary transformation. Overall, these tests will fill a knowledge gap in the scientific literature, and inform how future general-purpose emotion estimation systems are trained. In addition, the project will provide shared data that may lead to even greater advancements in the BCI field when used by future researchers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.