Influence of Temporal Difference Reward Prediction Errors on Brain Network Connectivity during Learning and Decision-Making

时间差异奖励预测误差对学习和决策过程中大脑网络连接的影响

• 批准号: 10374789
• 负责人: Lester Paul Sands
• 金额: $ 2.41万
• 依托单位: WAKE FOREST UNIVERSITY HEALTH SCIENCES
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-03-01 至 2022-05-21
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10374789
• 关键词: Address; Adult; Anatomy; Animals; Area; Back; Basal Ganglia; Behavior; Behavior Control; Behavior Disorders; Behavioral; Behavioral Symptoms; Brain; Brain region; Characteristics; Choice Behavior; Computer Models; Computing Methodologies; Corpus striatum structure; Coupling; Data; Decision Making; Disease; Electrophysiology (science); Experimental Designs; Feedback; Fellowship; Foundations; Functional Magnetic Resonance Imaging; Human; Impulse Control Disorders; Individual; Learning; Link; Magnetoencephalography; Maps; Measures; Mental Health; Methods; Modeling; National Research Service Awards; Participant; Pathologic; Pattern; Pharmaceutical Preparations; Plant Roots; Process; Psychological reinforcement; Psychophysics; Punishment; Research; Resolution; Rewards; Signal Transduction; Source; Stimulus; Substance abuse problem; Testing; Thalamencephalon; Time; Training; Update; adaptive learning; addiction; base; behavior measurement; blood oxygen level dependent; career; cingulate cortex; computational neuroscience; design; expectation; experience; experimental study; functional MRI scan; human model; innovation; insight; interest; learned behavior; learning outcome; multimodality; neural patterning; neurobehavioral; neuroimaging; reinforcer; relating to nervous system; response; reward circuitry; reward processing; skills; somatosensory; source localization; spatiotemporal; theories

PROJECT SUMMARY Disorders of choice behavior, such as substance abuse and impulse control disorders, involve over-valuing and repeatedly choosing certain reinforcers (e.g., drugs, risky actions), even in the face of recurring consequences. Understanding the link between adaptive choice behaviors and underlying neural activity is a strategic focus for substance abuse and mental health research. Much empirical evidence demonstrates that features of distinct pathological behaviors map onto distinct patterns of interactions between distributed brain regions. Nevertheless, how adaptive learning signals concerning rewards and punishments alter region-to-region functional interactions in real-time lies at the limit of our current understanding. As such, we seek to identify neurobehavioral markers that reflect how reinforcement learning (RL) signals alter functional brain network interactions and associated choice behaviors in healthy adults. Along this line of inquiry, this proposal’s central objective is to understand how real-time changes in inter-regional functional interactions – between, for instance, regions of the basal ganglia and limbic, prefrontal, and sensorimotor cortices – in response to RL signals influence adaptive choice behaviors. Our approach uses computational methods to investigate the quantitative relationship between measures of human choice behavior and brain network interactions at high-resolution spatiotemporal scales. Specifically, we pair computational RL models of human behavior on a probabilistic reward and punishment learning task with multi-modal functional neuroimaging to investigate changes in functional brain networks responsive to learning signals called ‘temporal difference reward prediction errors’ (TD RPEs). We will identify functional network interactions related to TD RPE signals to address, through two Specific Aims, our overarching hypothesis that TD RPE signals alter – in real time – the coupling (synchrony) of functional interactions between brain regions involved in processing rewards and punishments to direct changes in choice behavior. In Aim 1, we will measure functional networks interactions using magnetoencephalography (MEG) to test the hypotheses that (1) positive TD RPE signals increase, in real time, the coupling of inter-regional functional interactions and (2) negative or zero TD RPE signals decrease this coupling. In Aim 2, we use functional magnetic resonance imaging (fMRI) – within the same subjects from Aim 1 – to investigate individual-specific functional networks associated with TD RPE signals. Here, we will localize TD RPE-responsive functional regions-of-interest (ROIs) and incorporate these ROIs as individual-specific spatial priors for within-subject MEG analysis of functional network interactions. In all, this fellowship will provide training in contemporary modeling methods and experimental design that address fundamental questions in computational neuroscience regarding the functional organization of the human brain during adaptive learning. The central findings of this research will advance our understanding of the principles of brain activity supporting adaptive learning processes that are characteristically the root cause of many behavioral symptoms of substance abuse and related behavioral disorders.

项目总结 选择行为障碍，如药物滥用和冲动控制障碍，涉及高估和 反复选择某些增强剂(例如，药物、危险行为)，即使面对反复出现的后果。 理解适应性选择行为和潜在神经活动之间的联系是 药物滥用和心理健康研究。许多经验证据表明，不同的特征 病理行为映射到分布的大脑区域之间相互作用的不同模式。不过， 关于奖惩的适应性学习信号如何改变区域间的功能相互作用 实时存在于我们目前理解的极限。因此，我们试图确定神经行为标记物 反映了强化学习(RL)信号如何改变脑功能网络的相互作用和相关 健康成年人的选择行为。沿着这条调查路线，这项提案的中心目标是理解 区域间功能交互的实时变化--例如，在基础区域之间 神经节和边缘、前额叶和感觉运动皮质--对RL信号的反应影响适应性选择 行为。我们的方法使用计算方法来研究两者之间的定量关系 在高分辨率时空尺度上测量人类选择行为和大脑网络相互作用。 具体地说，我们将人类行为的计算RL模型与概率奖惩配对 用多模式功能神经成像研究脑功能网络变化的学习任务 对学习信号的反应称为“时间差奖励预测误差”(TD RPEs)。我们将确定 与TD RPE信号相关的功能网络交互通过两个具体目标解决我们的总体目标 假设TD RPE信号实时改变功能相互作用的耦合(同步) 大脑区域参与处理奖惩，以指导选择行为的变化。在目标1中， 我们将使用脑磁图(MEG)测量功能网络的相互作用，以验证假设 (1)正的TD RPE信号实时增加了区域间功能相互作用和 (2)负TD RPE信号或零TD RPE信号会降低这种耦合。在目标2中，我们使用了功能磁共振 成像(FMRI)-在来自AIM 1的相同受试者中-研究个体特定的功能网络 与TD RPE信号相关联。在这里，我们将本地化TD RPE响应功能感兴趣区域(ROI) 并将这些ROI合并为个体特定的空间先验，用于受试者内的功能性脑磁图分析 网络互动。总之，该奖学金将提供当代建模方法和 解决计算神经科学中有关功能的基本问题的实验设计 人脑在适应性学习过程中的组织。这项研究的主要发现将推动我们的 对大脑活动支持适应性学习过程的原理的理解 许多药物滥用和相关行为障碍的行为症状的根本原因。