Collaborative Research: Emotional Sophistication - Studies of Facial Expressions in Decision Making

即使在非常简单的博弈论情景中，社会和经济决策也不能完全用追求金钱利益最大化的“理性”尝试来解释。复杂的情绪过程，如愤怒、内疚或慷慨，都是导致可见行动的隐藏力量。这种“非理性”的动机可以驱动我们自己的决定，它们也会影响我们对是什么驱使他人做出决定的信念。这个项目的目标是使用动态面部表情的自动测量，结合其他测量，如功能磁共振成像(FMRI)和眼球跟踪，来调查非理性动机在社会决策中的作用。该方法的核心是使用最先进的计算机视觉技术，在参与者与计算机或彼此互动时，实时从视频中提取面部动作，在某些情况下，观看彼此面部的实时视频。研究人员将使用强大的统计机器学习技术来推断参与者在互动过程中的内部情绪状态。其目标是使用关于情绪状态的推理(A)预测参与者的行为；(B)解释为什么做出决定是根据驱动它的隐藏力量；以及(C)建立可以使用这些信息来驱动他们与人类互动的自主代理。这个多学科的项目为心理学、神经科学和经济学等多个领域做出了贡献。首先，它开发了研究决策过程的新方法。其次，它使用这些方法来检验关于社会决策的假设，并弥合可观察到的行动和产生这些行动的内部状态之间的差距。第三，研究人员打算提供一个数据集和工具集，这些数据集和工具集应该对其他研究人员在多种背景下分析面部表情非常有用。此外，对内部动机状态的自动和在线解码为“情感感知”交互计算机奠定了基础，即可以对用户的情绪和意图做出推断的人工系统。通过这些系统的开发，该项目将对日益增长的人机交互领域做出重大贡献。[由感知、行动和认知、决策、风险和管理科学以及稳健智能支持]

Social and economic decisions cannot be fully explained by "rational" attempts to maximize monetary gain, even in very simple game-theoretic scenarios. Complex emotional processes such as anger, guilt or generosity act as hidden forces that lead to observable actions. Such "non-rational" motivations can drive our own decisions and they affect our beliefs about what motivates others' decisions as well. The goal of this project is to use automatic measurements of dynamic facial expressions, in combination with other measurements such as functional MRI (fMRI) and eye-tracking, to investigate the role of non-rational motivations in social decision making. The core of the approach is to use state-of-the-art computer vision techniques to extract facial actions from video in real-time while participants interact with a computer or with each other, in some cases viewing live video of each others' faces. The investigators will use powerful statistical machine learning techniques to make inferences about the participants' internal emotional states during the interactions. The goal is to use the inferences concerning emotional state (a) to predict participants' behavior; (b) to explain why a decision is made in terms of the hidden forces driving it; and (c) to build autonomous agents that can use this information to drive their interactions with humans. This multidisciplinary project contributes to several fields such as psychology, neuroscience, and economics. First, it develops new methodologies to study decision processes. Second, it uses these methods to test hypotheses about social decision-making and to bridge the gap between observable actions and the internal states that generated them. Third, the investigators intend to make available a dataset and toolset that should be an extremely useful for other investigators analyzing facial expression in multiple contexts. Additionally, automatic and on-line decoding of internal motivational states lays the groundwork for "affectively-aware" interactive computers, or artificial systems that can make inferences about the emotions and intentions of their users. Through the development of these systems, this project will make a significant contribution to the growing field of human-machine interaction.[Supported by Perception, Action and Cognition, Decision, Risk and Management Sciences, and Robust Intelligence]