How Does Rumination Alter Learning and Choice Mechanisms in Reinforcement Learning?

反刍如何改变强化学习中的学习和选择机制？

• 批准号: 10549283
• 负责人: Peter Frank Hitchcock
• 金额: $ 5.52万
• 依托单位: BROWN UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-11-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10549283
• 关键词: Acceleration; Affect; Anxiety Disorders; Area; Attention; Basic Science; Bayesian Analysis; Bayesian Modeling; Behavior; Clinical; Computer Models; Conflict (Psychology); Data; Development; Diagnostic; Dissociation; Distress; Electroencephalography; Ensure; Experimental Designs; Generalized Anxiety Disorder; Goals; Impaired cognition; Impairment; Intervention; Interview; Learning; Link; Major Depressive Disorder; Mental Depression; Mental Health; Mental disorders; Mission; Modeling; National Institute of Mental Health; Occupations; Participant; Pathogenicity; Pattern; Phase; Prevention; Process; Productivity; Psychiatry; Psychological reinforcement; Public Health; Punishment; Recovery; Research; Research Personnel; Resources; Rewards; Short-Term Memory; Signal Transduction; Source; Stress; Testing; Thinking; Training; Translating; Variant; Visit; Work; career; cognitive control; cognitive task; computational neuroscience; expectation; experience; experimental study; improved; individualized medicine; neural; prevent; recruit; response; rumination; skills; stem; therapy development; translational research program

Rumination may promote ineffective behavior that ultimately harms mental health, although the mechanisms are unclear. Recent research suggests rumination, which is defined as dwelling on the possible significance and implications of one’s distress and problems, may promote ineffective behavior by disrupting reinforcement learning (RL). RL refers to the suite of processes that enable adjusting expectations, when an experience differs from expected, and then taking appropriate action. RL is a ubiquitous process that is necessary for effective behavior. Hence, rumination’s disruption of RL may represent an upstream process that helps explain rumination’s link to various ineffective behaviors. This raises the prospect of intervening to prevent rumination from disrupting RL, thereby making behavior more effective, and ultimately improving mental health. However, the recent research showing rumination disrupts RL is currently unclear on if rumination disrupts the RL mechanisms of learning (adjusting expectations) or choice (taking appropriate action). Different mechanisms would have distinct translational implications, hence clarity on the specific mechanisms is critically needed. Therefore, the objectives of this project are to dissociate rumination’s effects on the learning and choice components of RL, and to apply computational modeling and electroencephalography (EEG) to identify the specific mechanisms of these components it impairs. The central hypotheses are that rumination disrupts RL by impairing the following RL mechanisms: guidance by working memory that accelerates learning, long-term punishment learning, and cognitive control that enables effective choice. To test these hypotheses, the within- subject effects of ruminative (RUM) vs. neutral (NEU) thinking on mechanisms that guide learning (Aim 1) and choice (Aim 2) in RL will be investigated. To ensure generalization to clinical rumination, all participants (N=60) in this single-visit experimental study will meet criteria for Major Depressive Disorder or Generalized Anxiety Disorder. To ensure feasibility, recruitment will be embedded in a large-scale external project that conducts diagnostic interviews. Aim 1 will examine if RUM (vs. NEU) leads to slower acquisition of contingencies by disrupting working memory, and causes working memory to represent punishment experiences more strongly during initial learning, thereby disrupting long-term punishment learning. Aim 2 will examine if RUM (vs. NEU) disrupts cognitive control in a task where Pavlovian (instinctual) and instrumental (goal-directed) impulses conflict, and cognitive control is needed to override the former. These aims are integrated with a training plan to broaden and deepen the applicant’s background in computational neuroscience, to enable him to acquire specialized skills in this area (hierarchical Bayesian computational modeling and EEG), and to facilitate his development of a programmatic and competitive translational research program. This training is essential to the applicant’s transition to an independent, productive research career in the NIMH priority area of computational psychiatry.

沉思可能会促进无效的行为，最终损害心理健康，尽管其机制是 不清楚。最近的研究表明，沉思的定义是沉浸在可能的意义和 一个人的痛苦和问题的暗示，可能会通过破坏强化来促进无效的行为 学习(RL)。RL指的是能够在体验不同时调整预期的流程套件 出乎预期，然后采取适当的行动。RL是一个无处不在的过程，对于有效的 行为。因此，沉思对RL的破坏可能代表了一个上游过程，这有助于解释 沉思与各种无效的行为有关。这增加了干预以防止反省的可能性。 从而使行为更有效，并最终改善心理健康。然而， 最近的研究表明，沉思扰乱了RL，目前尚不清楚沉思是否会扰乱RL 学习机制(调整期望)或选择机制(采取适当行动)。不同的机制 将具有不同的翻译含义，因此迫切需要澄清具体机制。 因此，本项目的目标是分离沉思对学习和选择的影响 并应用计算建模和脑电(EEG)来识别 其损害这些成分的特定机制。核心假设是沉思通过以下方式扰乱RL 损害以下RL机制：通过加速学习的工作记忆进行指导，长期 惩罚学习和认知控制，使有效的选择成为可能。为了检验这些假设，内部- 反思性(朗姆酒)与中性(Neu)思维对引导学习机制的主体效应(目标1)和 RL中的选择(目标2)将被调查。为确保对临床反省的推广，所有参与者(N=60) 在这项单次访问的实验研究中，将满足严重抑郁障碍或广泛性焦虑的标准 无序。为了确保可行性，招聘将嵌入一个大型外部项目中，该项目进行 诊断性面谈。目标1将检查rum(vs.neu)是否通过以下方式导致较慢的意外情况获取 扰乱工作记忆，使工作记忆更强烈地代表惩罚经历 在最初的学习中，从而扰乱了长期的惩罚学习。目标2将检查朗姆酒(vs.neu) 在巴甫洛夫(本能)和工具性(目标导向)冲动的任务中扰乱认知控制 冲突，需要认知控制来超越前者。这些目标与培训计划相结合，以 拓宽和加深申请人在计算神经科学方面的背景，使他能够获得 这一领域的专门技能(分层贝叶斯计算建模和脑电)，并促进他的 制定具有计划性和竞争性的翻译研究计划。这项培训对于 申请者在NIMH计算优先领域向独立的、富有成效的研究生涯的过渡 精神病学。