How Does Rumination Alter Learning and Choice Mechanisms in Reinforcement Learning?

反刍如何改变强化学习中的学习和选择机制？

• 批准号: 10237171
• 负责人: Peter Frank Hitchcock
• 金额: $ 6.72万
• 依托单位: BROWN UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-11-01 至 2023-10-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10237171
• 关键词: Affect; Anxiety Disorders; Area; Attention; Basic Science; Bayesian Analysis; Bayesian Modeling; Behavior; Behavior Therapy; Clinical; Computer Models; Conflict (Psychology); Data; Development; Diagnostic; Distress; Electroencephalography; Ensure; Experimental Designs; Generalized Anxiety Disorder; Goals; Impaired cognition; Impairment; Intervention; Interview; Learning; Link; Major Depressive Disorder; Mental Depression; Mental Health; Mental disorders; Mission; Modeling; National Institute of Mental Health; Participant; Pathogenicity; Pattern; Phase; Plant Roots; Prevention; Process; Psychiatry; Psychological reinforcement; Public Health; Punishment; Recovery; Research; Research Personnel; Resources; Rewards; Short-Term Memory; Signal Transduction; Source; Stress; Testing; Thinking; Training; Translating; Variant; Visit; Work; base; career; cognitive control; cognitive task; computational neuroscience; expectation; experience; experimental study; improved; individualized medicine; prevent; recruit; relating to nervous system; response; rumination; skills; stem; therapy development; translational research program

Rumination may promote ineffective behavior that ultimately harms mental health, although the mechanisms are unclear. Recent research suggests rumination, which is defined as dwelling on the possible significance and implications of one’s distress and problems, may promote ineffective behavior by disrupting reinforcement learning (RL). RL refers to the suite of processes that enable adjusting expectations, when an experience differs from expected, and then taking appropriate action. RL is a ubiquitous process that is necessary for effective behavior. Hence, rumination’s disruption of RL may represent an upstream process that helps explain rumination’s link to various ineffective behaviors. This raises the prospect of intervening to prevent rumination from disrupting RL, thereby making behavior more effective, and ultimately improving mental health. However, the recent research showing rumination disrupts RL is currently unclear on if rumination disrupts the RL mechanisms of learning (adjusting expectations) or choice (taking appropriate action). Different mechanisms would have distinct translational implications, hence clarity on the specific mechanisms is critically needed. Therefore, the objectives of this project are to dissociate rumination’s effects on the learning and choice components of RL, and to apply computational modeling and electroencephalography (EEG) to identify the specific mechanisms of these components it impairs. The central hypotheses are that rumination disrupts RL by impairing the following RL mechanisms: guidance by working memory that accelerates learning, long-term punishment learning, and cognitive control that enables effective choice. To test these hypotheses, the within- subject effects of ruminative (RUM) vs. neutral (NEU) thinking on mechanisms that guide learning (Aim 1) and choice (Aim 2) in RL will be investigated. To ensure generalization to clinical rumination, all participants (N=60) in this single-visit experimental study will meet criteria for Major Depressive Disorder or Generalized Anxiety Disorder. To ensure feasibility, recruitment will be embedded in a large-scale external project that conducts diagnostic interviews. Aim 1 will examine if RUM (vs. NEU) leads to slower acquisition of contingencies by disrupting working memory, and causes working memory to represent punishment experiences more strongly during initial learning, thereby disrupting long-term punishment learning. Aim 2 will examine if RUM (vs. NEU) disrupts cognitive control in a task where Pavlovian (instinctual) and instrumental (goal-directed) impulses conflict, and cognitive control is needed to override the former. These aims are integrated with a training plan to broaden and deepen the applicant’s background in computational neuroscience, to enable him to acquire specialized skills in this area (hierarchical Bayesian computational modeling and EEG), and to facilitate his development of a programmatic and competitive translational research program. This training is essential to the applicant’s transition to an independent, productive research career in the NIMH priority area of computational psychiatry.

沉思可能会促进无效行为，最终损害心理健康，尽管其机制是 不清楚。最近的研究表明沉思，其定义是沉思可能的意义和 一个人的痛苦和问题的影响，可能会通过破坏强化而促进无效行为 学习（RL）。 RL 是指当体验不同时能够调整期望的一套流程 预期，然后采取适当的行动。强化学习是一个普遍存在的过程，对于有效的实施来说是必要的 行为。因此，反刍对强化学习的破坏可能代表了一个上游过程，有助于解释 沉思与各种无效行为的联系。这提出了进行干预以防止反省的可能性 避免破坏强化学习，从而使行为更加有效，并最终改善心理健康。然而， 最近的研究表明沉思会扰乱强化学习，目前尚不清楚沉思是否会扰乱强化学习 学习（调整期望）或选择（采取适当行动）的机制。机制不同 将会产生明显的转化影响，因此迫切需要明确具体机制。 因此，该项目的目标是分离沉思对学习和选择的影响 强化学习的组成部分，并应用计算建模和脑电图 (EEG) 来识别 它损害了这些组件的特定机制。核心假设是，沉思会通过以下方式破坏强化学习： 损害以下 RL 机制：加速学习的工作记忆引导、长期学习 惩罚学习和认知控制可以实现有效的选择。为了检验这些假设，内部 反刍式 (RUM) 与中立性 (NEU) 思维对指导学习机制的主体影响（目标 1）和 将研究 RL 中的选择（目标 2）。为了确保临床思考的泛化，所有参与者 (N=60) 在这项单次访问实验研究中，将满足重度抑郁症或广泛性焦虑症的标准 紊乱。为了确保可行性，招聘将被纳入一个大规模的外部项目中，该项目将进行 诊断访谈。目标 1 将检查 RUM（相对于 NEU）是否会导致意外事件的获取速度变慢 扰乱工作记忆，并导致工作记忆更强烈地代表惩罚经历 在初始学习期间，从而扰乱长期的惩罚学习。目标 2 将检查 RUM（相对于 NEU）是否 扰乱任务中的认知控制，其中巴甫洛夫（本能）和工具（目标导向）冲动 冲突，并且需要认知控制来克服前者。这些目标与培训计划相结合， 拓宽和加深申请人在计算神经科学方面的背景，使他能够获得 该领域的专业技能（分层贝叶斯计算建模和脑电图），并促进他 制定计划性和竞争性的转化研究计划。这项培训对于 申请人在 NIMH 计算优先领域过渡到独立、富有成效的研究生涯 精神病学。