Computational and brain predictors of emotion cue integration

该项目的目的是开发情绪线索整合的计算和基于大脑的模型： 人们基于动态的多模态线索对他人情绪的推断。观察者通常决定如何 目标的感觉基于诸如面部表情、韵律和语言的线索。这样的推论 健康的社会交往和不正常的推理都标志着并加剧了许多人的社会缺陷。 精神疾病心理学家和神经科学家研究情绪推理已经有几十年了， 绝大多数这类作品采用简化的社会线索，如小插曲或静态的人脸图像。通过 相反，“真实的世界”情感线索是复杂的、动态的和多模态的。基于线索集成推理 在认知和神经水平上，自然主义情绪信息可能不同于简单的推理，但这 这一现象仍然知之甚少。这意味着科学家们缺乏一个清晰的模型， 适应性地处理复杂的情绪线索，以及这种处理在精神疾病中是如何出错的。尤其 缺乏机制模型，可以描述的计算和大脑过程中涉及的线索， 具有足够精度的集成，以预测新情况、观察者和样本中的推断。该项目将 融合了社会心理学、计算机科学和神经科学的工具， 情绪线索整合的严格模型。我们已经证明了在面对复杂的情感时 提示、观察者动态地“加权”来自每种模态的提示（例如，视觉，语言）随着时间的推移，一个过程， (i)跟踪大脑活动和连接的变化;（ii）可以使用贝叶斯模型捕获。在这里，我们将 从几个方面扩展这项工作。首先，我们将开发精确的计算工具，以分离出 情感线索--如面部动作、韵律和语言情感--跟踪观察者对每一种情感的使用 整合过程中的线索模态。其次，我们将开发大脑活动的多区域“签名”， 在每种模态中跟踪情感推理的连接性。我们将使用这些签名与 机器学习来预测新观察者和样本中的单峰情感推断和线索整合， 仅仅基于大脑数据。第三，我们将探讨自然主义情绪推理的语境依赖性， 测试强化学习是否会使观察者的线索整合和伴随的大脑信号产生偏差。 最后，我们将模拟与患者的线索整合相关的计算和神经异常， 重度抑郁症和双相情感障碍在基础科学的层面上，这些数据将产生一个 从根本上讲，这是一种新的、更自然的情绪推理神经科学方法。的 我们生产的计算和大脑指标也将被用于促进开放和 实验室间情绪推理的累积研究。在翻译层面，我们将提供一个机械的，丰富的 情绪障碍中异常情绪推理的解释，为计算和大脑标记铺平了道路 可以用来评估这些和其他精神疾病的社会功能障碍和治疗效果。

The purpose of this project is to develop computational and brain-based models of emotion cue integration: people’s inferences about others’ emotions based on dynamic, multimodal cues. Observers often decide how targets feel based on cues such as facial expressions, prosody, and language. Such inferences scaffold healthy social interaction, and abnormal inference both marks and exacerbates social deficits in numerous psychiatric disorders. Psychologists and neuroscientists have studied emotion inference for decades, but the vast majority of this work employs simplified social cues, such as vignettes or static images of faces. By contrast, “real world” emotion cues are complex, dynamic, and multimodal. Cue integration—inference based on naturalistic emotion information—likely differs from simpler inference at cognitive and neural levels, but this phenomenon remains poorly understood. This means that scientists lack a clear model of how observers adaptively process complex emotion cues, and how that processing goes awry in mental illness. Especially lacking are mechanistic models that can describe the computations and brain processes involved in cue integration with sufficient precision to predict inference in new cases, observers, and samples. This project will merge tools from social psychology, computer science, and neuroscience to generate a novel and rigorous model of emotion cue integration. We have demonstrated that in the face of complex emotion cues, observers dynamically “weight” cues from each modality (e.g., visual, linguistic) over time, a process that (i) tracks shifts in brain activity and connectivity; and (ii) can be captured using Bayesian models. Here, we will expand this work in several ways. First, we will develop precise computational tools to isolate features of emotion cues—such as facial movements, prosody, and linguistic sentiment—that track observers’ use of each cue modality during integration. Second, we will develop multi-region “signatures” of brain activity and connectivity that track emotion inference in each modality. We will use these signatures in conjunction with machine learning to predict unimodal emotion inference and cue integration in new observers and samples, based on brain data alone. Third, we will explore the context-dependence of naturalistic emotion inference by testing whether reinforcement learning can bias observers’ cue integration and accompanying brain signatures. Finally, we will model computational and neural abnormalities associated with cue integration in patients with Major Depressive Disorder and Bipolar Disorder. At the level of basic science, these data will generate a fundamentally new—and more naturalistic—approach to the neuroscience of emotion inference. The computational and brain metrics we produce will also be made publically available to facilitate the open and cumulative study of emotion inference across labs. At a translational level, we will provide a mechanistic, rich account of abnormal emotion inference in mood disorders, paving the way for computational and brain markers that can be used to assess social dysfunction and treatment efficacy in these and other mental illnesses.