Dopaminergic mechanisms for motivation and reinforcement learning

摘要/摘要 这项研究计划试图描述和理解纹状体多巴胺(DA)信号，特别是在 伏隔核(NAC)。NAC DA是与动机和奖励相关的学习的关键调节器，并且 滥用药物具有增强NAC DA的共同属性。然而，我们缺乏一个清晰的解释，说明检察官是如何 信号产生，它们如何由本地NAC电路整形，以及它们如何调制NAC输出以影响 行为。我们在下一个资金阶段的目标是解决当前DA信号的三个具体方面 在激烈的辩论中： 首先，为什么NAC DA信号与纹状体其他部分的信号不同？我们发现南汽地区检察官 与背侧区域相比，进化较慢，并显示出对奖赏预测性的明显反应 暗示。我们假设，涉及NAC的神经回路产生奖励预测和动机 更长的时间范围。利用逆行光遗传标记，我们将比较个体DA的放电模式 投射到NAC亚区和其他纹状体区域的细胞，测试它们是否定义了临时地形图 奖励期望和反馈。 其次，NAC DA释放传递的信号是否超出了在传入DA细胞放电中编码的信号？ 我们发现，改变行为任务的可用奖励会影响老鼠的工作意愿，并且 改变NAC DA的释放，对DA细胞的激发没有任何明显的改变。我们将调查动机是否- 多巴胺的相关变化是由局部胆碱能中间神经元决定的。要做到这一点，我们将结合实际- 胆碱能的时间DA和ACh测量、选择性药理药物和光遗传标记 细胞。 第三，DA释放的增加如何迅速增强动力？这通常被认为涉及到 与表达D2受体的神经元相比，表达D1DA受体的NAC输出神经元的放电增强 受体。然而，最近的一些结果表明，这两个种群都增加了活动，而计算 模型对DA是否能足够快地调节细胞以解释观察到的行为表示怀疑 效果。为了解决这个问题，我们将记录已识别的NAC D1+和D2+输出神经元的尖峰，同时还 在关键的行为时刻测量和操纵DA。 在每一种情况下，我们都将利用光学化学传感器的最新技术进步，以及 我们记录自由活动大鼠个体识别神经元放电的能力。我们将使用多个 精心控制的行为任务，包括一个创新的新迷宫任务，在这个任务中，大鼠表现出权衡取舍 在期望的回报和需要的努力之间。总之，这些研究将为我们的 了解NAC DA功能，以及为研究社区提供的丰富的、公开可用的数据集。

Summary / Abstract This research program seeks to describe and understand striatal dopamine (DA) signals, particularly in the nucleus accumbens (NAc). NAc DA is a key regulator of both motivation and reward-related learning, and drugs of abuse share the common property of enhancing NAc DA. Yet we lack a clear account of how DA signals arise, how they are shaped by local NAc circuits, and how they modulate NAc output to influence behavior. Our Aims for the next funding period address three specific aspects of DA signals that are currently under active debate: First, why are NAc DA signals different to those in other parts of striatum? We found that NAc DA evolves more slowly compared to dorsal areas, and shows a distinct response profile to reward-predictive cues. We hypothesize that neural circuits involving NAc generate reward predictions and motivation over longer time horizons. Using retrograde optogenetic tagging, we will compare the firing patterns of individual DA cells that project to NAc subregions and other striatal areas, testing whether they define a temporal topography of reward expectation and feedback. Second, does NAc DA release convey signals beyond those encoded in the firing of afferent DA cells? We showed that changing the available reward for a behavioral task influences rats' willingness to work, and alters NAc DA release, without any apparent change in DA cell firing. We will investigate whether motivation- related changes in DA are instead sculpted by local cholinergic interneurons. To do this we will combine real- time DA and ACh measurements, selective pharmacological agents, and optogenetic tagging of cholinergic cells. Third, how do increases in DA release rapidly boost motivation? This is generally thought to involve enhanced firing of NAc output neurons that express the D1 DA receptor, relative to those expressing the D2 receptor. However, some recent results suggest that both populations increase activity, while computational models have cast doubt on whether DA can modulate cells quickly enough to explain observed behavioral effects. To resolve this we will record the spiking of identified NAc D1+ and D2+ output neurons, while also measuring and manipulating DA, at key behavioral moments. In each case we will take advantage of recent technical advances in optical chemical sensors, and of our ability to record the firing of individual identified neurons in freely-moving rats. We will use multiple carefully-controlled behavioral tasks, including an innovative new maze task in which rats display a trade-off between expected reward and required effort. Together these studies will provide vital new results for our understanding of NAc DA functions, along with rich, publicly-available data sets for the research community.