Computational and brain predictors of emotion cue integration

情绪线索整合的计算和大脑预测因子

• 批准号: 9923725
• 负责人: Jamil Zaki
• 金额: $ 45.18万
• 依托单位: STANFORD UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-05-19 至 2022-02-28
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9923725
• 关键词: Affect; Agreement; Base of the Brain; Basic Science; Bayesian Modeling; Bipolar Disorder; Brain; Brain region; Classification; Cognitive; Complex; Computer Models; Cues; Data; Dependence; Emotional; Emotional disorder; Emotions; Event; Exhibits; Face; Face Processing; Facial Expression; Functional Magnetic Resonance Imaging; Future; Image; Individual; Language; Lateral; Learning; Life; Linguistics; Machine Learning; Major Depressive Disorder; Measures; Mental disorders; Methods; Modality; Modeling; Mood Disorders; Moods; Movement; National Institute of Mental Health; Neurodevelopmental Disorder; Neurosciences; Observer Variation; Participant; Patients; Pattern; Perception; Process; Psychological reinforcement; Psychologist; Research; Research Domain Criteria; Research Personnel; Running; Sampling; Scanning; Scientist; Sensory; Social Functioning; Social Interaction; Social Processes; Social Psychology; Specific qualifier value; Speech; Structure; System; Testing; Time; Training; Treatment Efficacy; Visual; Weight; Work; affective computing; base; brain abnormalities; cognitive neuroscience; computer science; computerized tools; executive function; insight; language comprehension; multimodality; neuroimaging; novel; recruit; relating to nervous system; response; scaffold; social; social deficits; tool

The purpose of this project is to develop computational and brain-based models of emotion cue integration: people’s inferences about others’ emotions based on dynamic, multimodal cues. Observers often decide how targets feel based on cues such as facial expressions, prosody, and language. Such inferences scaffold healthy social interaction, and abnormal inference both marks and exacerbates social deficits in numerous psychiatric disorders. Psychologists and neuroscientists have studied emotion inference for decades, but the vast majority of this work employs simplified social cues, such as vignettes or static images of faces. By contrast, “real world” emotion cues are complex, dynamic, and multimodal. Cue integration—inference based on naturalistic emotion information—likely differs from simpler inference at cognitive and neural levels, but this phenomenon remains poorly understood. This means that scientists lack a clear model of how observers adaptively process complex emotion cues, and how that processing goes awry in mental illness. Especially lacking are mechanistic models that can describe the computations and brain processes involved in cue integration with sufficient precision to predict inference in new cases, observers, and samples. This project will merge tools from social psychology, computer science, and neuroscience to generate a novel and rigorous model of emotion cue integration. We have demonstrated that in the face of complex emotion cues, observers dynamically “weight” cues from each modality (e.g., visual, linguistic) over time, a process that (i) tracks shifts in brain activity and connectivity; and (ii) can be captured using Bayesian models. Here, we will expand this work in several ways. First, we will develop precise computational tools to isolate features of emotion cues—such as facial movements, prosody, and linguistic sentiment—that track observers’ use of each cue modality during integration. Second, we will develop multi-region “signatures” of brain activity and connectivity that track emotion inference in each modality. We will use these signatures in conjunction with machine learning to predict unimodal emotion inference and cue integration in new observers and samples, based on brain data alone. Third, we will explore the context-dependence of naturalistic emotion inference by testing whether reinforcement learning can bias observers’ cue integration and accompanying brain signatures. Finally, we will model computational and neural abnormalities associated with cue integration in patients with Major Depressive Disorder and Bipolar Disorder. At the level of basic science, these data will generate a fundamentally new—and more naturalistic—approach to the neuroscience of emotion inference. The computational and brain metrics we produce will also be made publically available to facilitate the open and cumulative study of emotion inference across labs. At a translational level, we will provide a mechanistic, rich account of abnormal emotion inference in mood disorders, paving the way for computational and brain markers that can be used to assess social dysfunction and treatment efficacy in these and other mental illnesses.

该项目的目的是开发情感线索整合的计算和基于大脑的模型： 人们基于动态、多模式线索对其他人情绪的推断。观察者经常决定如何 目标感觉基于面部表情、韵律和语言等线索。这样的推论脚手架 健康的社交互动和异常的推理都标志着并加剧了许多人的社交缺陷 精神疾病。心理学家和神经科学家研究情绪推理已有数十年，但 这项作品的绝大多数都采用了简化的社交线索，例如小插图或面部静态图像。经过 相比之下，“现实世界”的情感线索是复杂的、动态的、多模式的。线索整合——基于推理 关于自然主义情感信息——可能与认知和神经水平上的简单推理不同，但这 现象仍然知之甚少。这意味着科学家缺乏一个清晰的模型来说明观察者如何 适应性地处理复杂的情绪线索，以及这种处理在精神疾病中如何出错。尤其 缺乏可以描述提示所涉及的计算和大脑过程的机械模型 以足够的精度进行积分，以预测新案例、观察者和样本中的推理。该项目将 融合社会心理学、计算机科学和神经科学的工具，产生新颖且 严格的情感线索整合模型。我们已经证明，面对复杂的情绪 随着时间的推移，观察者动态地“衡量”来自每种模态（例如视觉、语言）的线索，这一过程 (i) 追踪大脑活动和连通性的变化； (ii) 可以使用贝叶斯模型捕获。在这里，我们将 以多种方式扩展这项工作。首先，我们将开发精确的计算工具来分离特征 情感线索——例如面部动作、韵律和语言情感——跟踪观察者对每个内容的使用 整合期间的提示方式。其次，我们将开发大脑活动的多区域“特征” 跟踪每种模式中的情感推理的连接性。我们将结合使用这些签名 机器学习来预测新观察者和样本中的单峰情感推理和线索整合， 仅基于大脑数据。第三，我们将探讨自然主义情感推理的上下文依赖性 测试强化学习是否会使观察者的线索整合和伴随的大脑特征产生偏差。 最后，我们将模拟与线索整合相关的计算和神经异常患者 重度抑郁症和双相情感障碍。在基础科学层面，这些数据将产生 情绪推理神经科学的全新且更自然的方法。这 我们生产的计算和大脑指标也将公开发布，以促进开放和 跨实验室情感推断的累积研究。在翻译层面，我们将提供机械化、丰富的 解释情绪障碍中的异常情绪推断，为计算和大脑标记铺平道路 可用于评估这些和其他精神疾病的社会功能障碍和治疗效果。