Reinforcement Learning and Striatal Patch/Matrix Architecture

描述（由申请人提供）：滥用药物可以引起持久的行为变化，部分原因是异常参与神经可塑性机制，这些机制是正常的认知驱动学习的基础。特别是，药物成瘾的关键特征可能来自处理行为反应的渐进自动化的背侧纹状体回路的功能改变。纹状体回路被广泛认为使用与人工强化学习（RL）算法基本相似的原理进行自适应决策。然而，这些电路的特定组件如何映射到特定的计算/行为功能仍然存在争议。在背侧纹状体内，存在称为“斑块”（或纹状体）的显著亚区，其与周围的“基质”具有非常不同的连接性和神经化学。长期以来，人们一直假设补丁在强化学习中有特殊的作用，有助于控制对行为后果的评估反馈。然而，测试这些想法是不可能的，主要是由于区分行为动物中的补丁和基质神经元的技术限制。我们现在能够克服这一关键障碍，并大大推进对奖励体验如何导致行为改变的理解。本申请的目标是a）完成新一代电生理探针的开发，用于从识别的纹状体位置进行高密度记录，以及B）在强化学习任务期间使用这些设备比较贴片和基质神经元的活动模式。通过这种方式，我们将测试特定的假设，即补丁神经元编码与奖励预测相关的信号。对纹状体区室在强化学习中的作用的准确描述将极大地有助于调查滥用药物如何劫持正常决策。此外，新设备将广泛应用于研究纹状体和其他神经回路中的神经编码。

DESCRIPTION (provided by applicant): Drugs of abuse can provoke long-lasting behavioral change, in part by abnormally engaging neural plasticity mechanisms that underlie normal reinforcement-driven learning. In particular, critical features of drug addiction may arise from altered function of dorsal striatal circuits that handle the progressive automatization of behavioral responses. Striatal circuits are widely thought to operate using essentially similar principles as artificial reinforcement learning (RL) algorithms for adaptive decision-making. However, how specific components of these circuits map onto specific computational/behavioral functions remains controversial. Within dorsal striatum there are remarkable subregions called "patches" (or striosomes) that have very different connectivity and neurochemistry to the surrounding "matrix". It has long been hypothesized that patches have a special role in reinforcement learning, helping to control evaluative feedback about the consequences of actions. However, testing such ideas has not been possible, largely due to technical limitations in distinguishing patch and matrix neurons in behaving animals. We are now in a position to overcome this critical barrier to progress, and greatly advance understanding of just how rewarding experiences lead to altered behavior. The goals of this application are a) to complete development of a new generation of electrophysiological probes for high-density recording from identified striatal locations, and b) to use these devices to compare the activity patterns of patch and matrix neurons during reinforcement learning tasks. In this way we will test the specific hypothesis that patch neurons encode signals related to reward prediction. An accurate description of the roles of striatal compartments in reinforcement learning would greatly assist investigations into how abused drugs hijack normal decision-making. In addition, the new devices would be of broad application for investigating neural coding in both striatum and other neural circuits.