Reinforcement Learning and Striatal Patch/Matrix Architecture

描述（由申请人提供）：药物滥用可以引起长期的行为改变，部分原因是异常参与的神经可塑性机制是正常强化驱动学习的基础。特别是，药物成瘾的关键特征可能来自处理行为反应的渐进式自动化的背纹状体回路的功能改变。纹状体回路被广泛认为使用与人工强化学习（RL）算法本质上相似的原理进行自适应决策。然而，这些电路的特定组件如何映射到特定的计算/行为功能仍然存在争议。在背纹状体中，有显著的亚区称为“斑块”（或纹状体），它们与周围的“基质”具有非常不同的连通性和神经化学。长期以来，人们一直假设补丁在强化学习中具有特殊作用，有助于控制有关行为后果的评估反馈。然而，测试这些想法是不可能的，主要是由于在区分行为动物的斑块和基质神经元方面的技术限制。我们现在有能力克服这个阻碍进步的关键障碍，并极大地推进对奖励体验如何导致行为改变的理解。本应用程序的目标是a)完成新一代电生理探针的开发，用于从确定的纹状体位置进行高密度记录，b)使用这些设备来比较强化学习任务期间补丁和基质神经元的活动模式。通过这种方式，我们将测试补丁神经元编码与奖励预测相关的信号的特定假设。准确描述纹状体隔室在强化学习中的作用，将极大地有助于调查滥用药物是如何劫持正常决策的。此外，新装置将广泛应用于纹状体和其他神经回路的神经编码研究。

DESCRIPTION (provided by applicant): Drugs of abuse can provoke long-lasting behavioral change, in part by abnormally engaging neural plasticity mechanisms that underlie normal reinforcement-driven learning. In particular, critical features of drug addiction may arise from altered function of dorsal striatal circuits that handle the progressive automatization of behavioral responses. Striatal circuits are widely thought to operate using essentially similar principles as artificial reinforcement learning (RL) algorithms for adaptive decision-making. However, how specific components of these circuits map onto specific computational/behavioral functions remains controversial. Within dorsal striatum there are remarkable subregions called "patches" (or striosomes) that have very different connectivity and neurochemistry to the surrounding "matrix". It has long been hypothesized that patches have a special role in reinforcement learning, helping to control evaluative feedback about the consequences of actions. However, testing such ideas has not been possible, largely due to technical limitations in distinguishing patch and matrix neurons in behaving animals. We are now in a position to overcome this critical barrier to progress, and greatly advance understanding of just how rewarding experiences lead to altered behavior. The goals of this application are a) to complete development of a new generation of electrophysiological probes for high-density recording from identified striatal locations, and b) to use these devices to compare the activity patterns of patch and matrix neurons during reinforcement learning tasks. In this way we will test the specific hypothesis that patch neurons encode signals related to reward prediction. An accurate description of the roles of striatal compartments in reinforcement learning would greatly assist investigations into how abused drugs hijack normal decision-making. In addition, the new devices would be of broad application for investigating neural coding in both striatum and other neural circuits. PUBLIC HEALTH RELEVANCE: This project aims for a better understanding of the neural mechanisms underlying adaptive and maladaptive decision-making. Success in this project may assist the design of novel therapies for human disorders that are characterized by inappropriate choices, especially drug addiction.