Collaborative Research: Emotional Sophistication - Studies of Facial Expressions in Decision Making

即使在非常简单的博弈论场景中，社会和经济决策也不能完全用“理性的”货币收益最大化的尝试来解释。复杂的情绪过程，如愤怒、内疚或慷慨，是导致可观察行为的隐藏力量。这种“非理性”的动机可以驱动我们自己的决定，也会影响我们对他人决定动机的看法。这个项目的目标是使用动态面部表情的自动测量，结合其他测量，如功能磁共振成像（fMRI）和眼球追踪，来研究非理性动机在社会决策中的作用。该方法的核心是使用最先进的计算机视觉技术，在参与者与计算机或彼此互动时，实时从视频中提取面部动作，在某些情况下，观看彼此面部的实时视频。研究人员将使用强大的统计机器学习技术来推断参与者在互动过程中的内部情绪状态。目标是使用有关情绪状态的推论(a)来预测参与者的行为；(b)解释作出决定的原因及背后的推动力；(c)建立能够利用这些信息驱动它们与人类互动的自主代理。这个多学科项目涉及心理学、神经科学和经济学等多个领域。首先，它开发了新的方法来研究决策过程。其次，它使用这些方法来检验关于社会决策的假设，并弥合可观察到的行为与产生这些行为的内部状态之间的差距。第三，研究人员打算提供一个数据集和工具集，这对于其他研究人员在多种情况下分析面部表情非常有用。此外，内部动机状态的自动和在线解码为“情感感知”交互式计算机或能够推断用户情感和意图的人工系统奠定了基础。通过这些系统的开发，该项目将对不断发展的人机交互领域做出重大贡献。[感知科学、行动与认知科学、决策科学、风险与管理科学、稳健智力科学支持]

Social and economic decisions cannot be fully explained by "rational" attempts to maximize monetary gain, even in very simple game-theoretic scenarios. Complex emotional processes such as anger, guilt or generosity act as hidden forces that lead to observable actions. Such "non-rational" motivations can drive our own decisions and they affect our beliefs about what motivates others' decisions as well. The goal of this project is to use automatic measurements of dynamic facial expressions, in combination with other measurements such as functional MRI (fMRI) and eye-tracking, to investigate the role of non-rational motivations in social decision making. The core of the approach is to use state-of-the-art computer vision techniques to extract facial actions from video in real-time while participants interact with a computer or with each other, in some cases viewing live video of each others' faces. The investigators will use powerful statistical machine learning techniques to make inferences about the participants' internal emotional states during the interactions. The goal is to use the inferences concerning emotional state (a) to predict participants' behavior; (b) to explain why a decision is made in terms of the hidden forces driving it; and (c) to build autonomous agents that can use this information to drive their interactions with humans. This multidisciplinary project contributes to several fields such as psychology, neuroscience, and economics. First, it develops new methodologies to study decision processes. Second, it uses these methods to test hypotheses about social decision-making and to bridge the gap between observable actions and the internal states that generated them. Third, the investigators intend to make available a dataset and toolset that should be an extremely useful for other investigators analyzing facial expression in multiple contexts. Additionally, automatic and on-line decoding of internal motivational states lays the groundwork for "affectively-aware" interactive computers, or artificial systems that can make inferences about the emotions and intentions of their users. Through the development of these systems, this project will make a significant contribution to the growing field of human-machine interaction.[Supported by Perception, Action and Cognition, Decision, Risk and Management Sciences, and Robust Intelligence]