Reinforcement Learning and Striatal Patch/Matrix Architecture

描述(由申请人提供)：滥用药物可以引起长期的行为变化，部分原因是异常地参与了正常强化驱动学习的神经可塑性机制。特别是，药物成瘾的关键特征可能源于处理行为反应进行性自动化的背侧纹状体回路的功能改变。纹状体回路被广泛认为使用与人工强化学习(RL)算法基本相似的原理进行自适应决策。然而，这些电路的特定组件如何映射到特定的计算/行为功能仍然存在争议。在背侧纹状体中，有一些显著的亚区，称为“斑块”(或纹状体)，它们与周围的“基质”有着截然不同的连接和神经化学。长期以来，人们一直假设，补丁在强化学习中具有特殊的作用，有助于控制关于行为后果的评价性反馈。然而，测试这种想法是不可能的，主要是因为在区分行为动物的斑块神经元和基质神经元方面存在技术限制。我们现在能够克服这一关键的进步障碍，并极大地促进对有益经历如何导致行为改变的理解。这项应用的目标是a)完成新一代电生理探针的开发，用于从已识别的纹状体位置进行高密度记录，以及b)使用这些设备来比较强化学习任务中斑块和基质神经元的活动模式。通过这种方式，我们将检验特定的假设，即斑块神经元编码与奖励预测相关的信号。准确描述纹状体隔区在强化学习中的作用将极大地有助于调查滥用药物是如何劫持正常决策的。此外，这些新设备将在研究纹状体和其他神经回路的神经编码方面具有广泛的应用。 公共卫生相关性：该项目旨在更好地理解适应性和非适应性决策背后的神经机制。该项目的成功可能有助于为人类疾病设计新的治疗方法，这些疾病的特点是选择不当，特别是吸毒成瘾。

DESCRIPTION (provided by applicant): Drugs of abuse can provoke long-lasting behavioral change, in part by abnormally engaging neural plasticity mechanisms that underlie normal reinforcement-driven learning. In particular, critical features of drug addiction may arise from altered function of dorsal striatal circuits that handle the progressive automatization of behavioral responses. Striatal circuits are widely thought to operate using essentially similar principles as artificial reinforcement learning (RL) algorithms for adaptive decision-making. However, how specific components of these circuits map onto specific computational/behavioral functions remains controversial. Within dorsal striatum there are remarkable subregions called "patches" (or striosomes) that have very different connectivity and neurochemistry to the surrounding "matrix". It has long been hypothesized that patches have a special role in reinforcement learning, helping to control evaluative feedback about the consequences of actions. However, testing such ideas has not been possible, largely due to technical limitations in distinguishing patch and matrix neurons in behaving animals. We are now in a position to overcome this critical barrier to progress, and greatly advance understanding of just how rewarding experiences lead to altered behavior. The goals of this application are a) to complete development of a new generation of electrophysiological probes for high-density recording from identified striatal locations, and b) to use these devices to compare the activity patterns of patch and matrix neurons during reinforcement learning tasks. In this way we will test the specific hypothesis that patch neurons encode signals related to reward prediction. An accurate description of the roles of striatal compartments in reinforcement learning would greatly assist investigations into how abused drugs hijack normal decision-making. In addition, the new devices would be of broad application for investigating neural coding in both striatum and other neural circuits. PUBLIC HEALTH RELEVANCE: This project aims for a better understanding of the neural mechanisms underlying adaptive and maladaptive decision-making. Success in this project may assist the design of novel therapies for human disorders that are characterized by inappropriate choices, especially drug addiction.