How Does Rumination Alter Learning and Choice Mechanisms in Reinforcement Learning?

沉思可能会促进无效的行为，最终损害心理健康，尽管其机制是 不清楚最近的研究表明，反刍，这是定义为居住在可能的意义， 一个人的痛苦和问题的影响，可能会促进无效的行为，通过破坏强化 学习（RL）。RL指的是当体验不同时，能够调整期望的一套过程 然后采取适当的行动。强化学习是一个无处不在的过程， 行为因此，反刍对RL的破坏可能代表了一个上游过程，有助于解释 反刍与各种无效行为的联系。这提出了干预以防止反刍的前景 从破坏RL，从而使行为更有效，并最终改善心理健康。然而，在这方面， 最近的研究表明，反刍干扰RL目前还不清楚反刍是否会干扰RL 学习机制（调整期望）或选择（采取适当的行动）。不同的机制 将具有不同的翻译影响，因此迫切需要明确具体机制。 因此，本研究的目的是分离反刍对学习和选择的影响 RL的组成部分，并应用计算建模和脑电图（EEG）来识别 它损害的这些组件的特定机制。核心假设是反刍通过以下方式破坏RL： 损害以下RL机制：工作记忆加速学习的指导，长期 惩罚学习和认知控制，使有效的选择。为了验证这些假设，内部- 反刍（RUM）与中性（NEU）思维对指导学习机制的影响（目标1）， 将研究RL中的选择（目标2）。为了确保推广到临床反刍，所有参与者（N=60） 在这项单次访问的实验性研究中，将符合严重抑郁症或广泛性焦虑症的标准 Disorder.为确保可行性，招聘工作将纳入一个大规模的外部项目， 诊断面试目标1将检查RUM（相对于NEU）是否会导致更慢的意外事件获取， 扰乱工作记忆，并导致工作记忆更强烈地代表惩罚经历 在最初的学习过程中，从而破坏了长期的惩罚学习。目标2将检查RUM（与NEU） 在巴甫洛夫（本能）和工具（目标导向）冲动的任务中， 冲突，需要认知控制来推翻前者。这些目标与培训计划相结合， 拓宽和加深申请人在计算神经科学方面的背景，使他能够获得 专业技能在这一领域（分层贝叶斯计算建模和脑电图），并促进他的 制定一个有计划和有竞争力的翻译研究计划。这种培训对于 申请人的过渡到一个独立的，富有成效的研究生涯在NIMH优先领域的计算 精神病学

Rumination may promote ineffective behavior that ultimately harms mental health, although the mechanisms are unclear. Recent research suggests rumination, which is defined as dwelling on the possible significance and implications of one’s distress and problems, may promote ineffective behavior by disrupting reinforcement learning (RL). RL refers to the suite of processes that enable adjusting expectations, when an experience differs from expected, and then taking appropriate action. RL is a ubiquitous process that is necessary for effective behavior. Hence, rumination’s disruption of RL may represent an upstream process that helps explain rumination’s link to various ineffective behaviors. This raises the prospect of intervening to prevent rumination from disrupting RL, thereby making behavior more effective, and ultimately improving mental health. However, the recent research showing rumination disrupts RL is currently unclear on if rumination disrupts the RL mechanisms of learning (adjusting expectations) or choice (taking appropriate action). Different mechanisms would have distinct translational implications, hence clarity on the specific mechanisms is critically needed. Therefore, the objectives of this project are to dissociate rumination’s effects on the learning and choice components of RL, and to apply computational modeling and electroencephalography (EEG) to identify the specific mechanisms of these components it impairs. The central hypotheses are that rumination disrupts RL by impairing the following RL mechanisms: guidance by working memory that accelerates learning, long-term punishment learning, and cognitive control that enables effective choice. To test these hypotheses, the within- subject effects of ruminative (RUM) vs. neutral (NEU) thinking on mechanisms that guide learning (Aim 1) and choice (Aim 2) in RL will be investigated. To ensure generalization to clinical rumination, all participants (N=60) in this single-visit experimental study will meet criteria for Major Depressive Disorder or Generalized Anxiety Disorder. To ensure feasibility, recruitment will be embedded in a large-scale external project that conducts diagnostic interviews. Aim 1 will examine if RUM (vs. NEU) leads to slower acquisition of contingencies by disrupting working memory, and causes working memory to represent punishment experiences more strongly during initial learning, thereby disrupting long-term punishment learning. Aim 2 will examine if RUM (vs. NEU) disrupts cognitive control in a task where Pavlovian (instinctual) and instrumental (goal-directed) impulses conflict, and cognitive control is needed to override the former. These aims are integrated with a training plan to broaden and deepen the applicant’s background in computational neuroscience, to enable him to acquire specialized skills in this area (hierarchical Bayesian computational modeling and EEG), and to facilitate his development of a programmatic and competitive translational research program. This training is essential to the applicant’s transition to an independent, productive research career in the NIMH priority area of computational psychiatry.