Reversal Learning and Dopamine: A Bayesian Perspective
Reversal Learning and Dopamine: A Bayesian Perspective
复制标题
DOI：
10.1523/jneurosci.1989-14.2015
复制
发表时间：
2015-02-11
影响因子：
5.3
通讯作者：
Averbeck, Bruno B.
中科院分区：
文献类型：
--
作者：
Costa, Vincent D.;Tran, Valery L.;Averbeck, Bruno B.
抑制性学习被研究为学习抑制先前奖励的行为的过程。逆转学习的缺陷已被视为多巴胺和眶额皮质损伤后的操作。然而，逆转学习通常在对逆转经验有限的动物中进行研究。因此，动物正在学习在数据收集期间发生逆转。我们已经研究了一个任务制度，其中猴子有丰富的经验与逆转和稳定的行为表现的概率双臂强盗逆转学习任务。我们开发了一种贝叶斯分析方法，以检查在这个制度中的多巴胺的操作对逆转性能的影响。我们发现，分析可以澄清动物的策略。具体来说，在反转时，猴子会迅速从选择一种刺激切换到选择另一种刺激，而不是逐渐过渡，如果它们使用朴素的强化学习（RL）更新值，这可能是预期的。此外，我们发现，氟哌啶醇的管理影响的方式，动物整合到他们的选择行为的先验知识。与左旋多巴（L-DOPA）或安慰剂相比，动物对氟哌啶醇逆转发生的位置有更强的先验。这种强先验是适当的，因为动物对块中间发生的反转有丰富的经验。总体而言，我们发现，贝叶斯解剖的行为澄清了动物的策略，并揭示了氟哌啶醇的影响，有利于选择逆转的证据与先验信息的整合。
Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (L-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.