Neuronal networks for control of eye movement

动机控制中的多巴胺-奖励，厌恶和警觉：中脑多巴胺神经元以其对奖励的强烈反应和在积极动机中的关键作用而闻名。然而，越来越清楚的是，多巴胺神经元也会传递与显著但非奖励性的体验相关的信号，比如厌恶和警觉事件。在这里，我们回顾了最近的进展，了解多巴胺的奖励和非奖励功能。基于这些数据，我们提出多巴胺神经元有多种类型，它们与不同的大脑网络相连，在动机控制中扮演不同的角色。一些多巴胺神经元编码动机价值，支持大脑网络寻求，评估和价值学习。另一些编码动机的突出性，支持大脑网络的定向，认知和一般动机。这两种类型的多巴胺神经元都被一种警报信号所增强，这种警报信号参与了对潜在的重要感觉线索的快速检测。我们假设，这些多巴胺能通路的价值，显着性和警觉合作，以支持适应性行为。 多巴胺介导的学习和皮质-纹状体回路的转换解释了强化学习中的行为变化：基底神经节被认为在强化学习中起着至关重要的作用。学习机制的核心是位于皮质-纹状体突触中的多巴胺（DA）D1和D2受体。然而，目前还不清楚这种DA介导的突触可塑性是如何部署和协调奖励条件性行为的变化。在这里，我们提出了一个强化学习的计算模型，使用不同的阈值D1和D2介导的突触可塑性，DA独立的突触可塑性拮抗。由大于预期的奖励引起的DA释放的阶段性增加在直接通路中诱导长时程增强（LTP），而由小于预期的奖励引起的DA释放的阶段性减少在间接通路中诱导长时程抑制的停止，导致LTP。这种学习机制可以解释在位置-奖励-值-关联任务中观察到的鲁棒行为适应，其中动物对奖励位置进行较短的潜伏期扫视。随着猴子变得更有经验，扫视潜伏期的变化变得更快。这种行为可以通过选择性激活皮质-纹状体回路的转换机制来解释。我们的模型还显示了D1或D2受体阻断实验如何选择性地影响奖励或无奖励试验。提出的机制也解释了帕金森病的行为变化。 克服先天、习惯和动机行为的皮质-基底神经节机制：大多数人类行为是在熟悉的环境下自动执行的。这些行为的优势在于它们优先于任何其他潜在的选择。然而，人类也有能力利用认知资源来抑制这种优势行为，并在不可预见的情况下用另一种受控行为来取代它。这种在短时间内转换行为的非凡能力是执行功能的标志。在这篇文章中，我们首先认为，优势自动性至少可以出现在三个不同的领域-先天，习惯和动机。然后，我们回顾了神经生理学的发现，大脑如何实现其开关功能，在每个领域，主要是通过专注于猴子眼神经系统作为实验模型。新出现的证据表明，在共享的皮质基底神经节网络中的多个神经元群体有助于压倒性的优势眼球运动，无论是先天的，习惯性的还是动机性的。这种考虑表明，皮质-基底神经节网络作为人类和其他动物防止自己屈从于反射、习惯和动机冲动的神经机制具有普遍的多功能性。

Dopamine in motivational control - rewarding, aversive, and alerting: Midbrain dopamine neurons are well known for their strong responses to rewards and their critical role in positive motivation. It has become increasingly clear, however, that dopamine neurons also transmit signals related to salient but nonrewarding experiences such as aversive and alerting events. Here we review recent advances in understanding the reward and nonreward functions of dopamine. Based on this data, we propose that dopamine neurons come in multiple types that are connected with distinct brain networks and have distinct roles in motivational control. Some dopamine neurons encode motivational value, supporting brain networks for seeking, evaluation, and value learning. Others encode motivational salience, supporting brain networks for orienting, cognition, and general motivation. Both types of dopamine neurons are augmented by an alerting signal involved in rapid detection of potentially important sensory cues. We hypothesize that these dopaminergic pathways for value, salience, and alerting cooperate to support adaptive behavior. Dopamine-mediated learning and switching in cortico-striatal circuit explain behavioral changes in reinforcement learning: The basal ganglia are thought to play a crucial role in reinforcement learning. Central to the learning mechanism are dopamine (DA) D1 and D2 receptors located in the cortico-striatal synapses. However, it is still unclear how this DA-mediated synaptic plasticity is deployed and coordinated during reward-contingent behavioral changes. Here we propose a computational model of reinforcement learning that uses different thresholds of D1- and D2-mediated synaptic plasticity which are antagonized by DA-independent synaptic plasticity. A phasic increase in DA release caused by a larger-than-expected reward induces long-term potentiation (LTP) in the direct pathway, whereas a phasic decrease in DA release caused by a smaller-than-expected reward induces a cessation of long-term depression, leading to LTP in the indirect pathway. This learning mechanism can explain the robust behavioral adaptation observed in a location-reward-value-association task where the animal makes shorter latency saccades to reward locations. The changes in saccade latency become quicker as the monkey becomes more experienced. This behavior can be explained by a switching mechanism which activates the cortico-striatal circuit selectively. Our model also shows how D1- or D2-receptor blocking experiments affect selectively either reward or no-reward trials. The proposed mechanisms also explain the behavioral changes in Parkinsons disease. Cortico-basal ganglia mechanisms for overcoming innate, habitual and motivational behaviors: Most of the human behaviors are executed automatically under familiar circumstances. These behaviors are prepotent in that they take precedence over any other potential alternatives. Yet, humans are also capable of engaging cognitive resources to inhibit such a prepotent behavior and replace it with an alternative controlled behavior in response to an unforeseen situation. This remarkable capability to switch behaviors in a short period of time is the hallmark of executive functions. In this article, we first argue that the prepotent automaticity could emerge at least in three different domains - innate, habitual and motivational. We then review neurophysiological findings on how the brain might realize its switching functions in each domain, primarily by focusing on the monkey oculomotor system as the experimental model. Emerging evidence now suggests that multiple neuronal populations in the shared cortico-basal ganglia network contribute to overriding prepotent eye movement, be its origin innate, habitual or motivational. This consideration suggests the general versatility of the cortico-basal ganglia network as the neural mechanism whereby humans and other animals keep themselves from becoming subservient to reflex, habit and motivational impulses.