Does prefrontal dopamine modulate error signals to optimally adjust learning?

描述（由申请人提供）：人类和动物学会根据过去的经验有效地选择行动。强化学习的一种特殊形式，涉及从预测奖励的错误中学习，为广泛的学习现象提供了简洁的解释。这些模型也提供了一些见解的生物机制参与这一进程。投射到纹状体的多巴胺神经元被认为编码了一种“奖励预测误差”，用于训练纹状体中的神经元，以反映特定状态下特定动作的价值。虽然传统的强化学习模型 简单而有效，它们未能捕捉到人类学习行为的至少一个显著方面： 人们从某些错误中学到的东西比从其他错误中学到的要多。特别是，人们往往更容易受到错误的影响，这些错误是如此突出，以至于暗示了上下文的变化或在不确定的时刻发生的错误。最优推理的抽象统计模型很好地描述了这种行为，但它在大脑中实现的机制仍然未知。 在这里，我研究了一种潜在的机制，通过这种机制，学习的合理调整可能会在大脑中实现：前扣带皮层（ACC），大脑中对行为更新很重要的一个区域，可能代表当前的上下文，并将此信息传递给纹状体中编码动作值的神经元。通过在显著错误之后表示新的上下文，ACC可以驱动一组新的纹状体神经元的激活，从而丢弃在先前上下文中收集的不相关信息并加速学习。虽然这样的系统允许在学习中进行合理的调整，但它将需要对ACC中的上下文表示的维护和丢弃进行非常精细的调整控制。更高的紧张性多巴胺水平被认为提高了网络稳定性，在ACC中，可能导致稳定的上下文表示和针对稳定环境优化的学习速率。这个建议的目标是为我提供计算建模，人类脑电图测量和行为药理学方面的培训。这种训练使我能够检验这一假设，即ACC中的多巴胺能神经调节系统和网络通过两个特定的目的在调节结果对未来行动的影响方面发挥互补作用。第一个目标将检查反馈锁定的EEG响应是否来自ACC反映学习的合理调整，预测行为更新，并与上下文表示的变化相一致。第二个目标将检查是否快速增加皮质多巴胺水平减缓学习和减轻反馈锁定EEG反应。

DESCRIPTION (provided by applicant): Humans and animals learn to effectively select actions based on past experience. One particular form of reinforcement learning that involves learning from errors in predicting rewards has provided parsimonious explanations for a broad range of learning phenomena. Such models have also provided some insights into the biological machinery involved in this process. Dopamine neurons projecting to the striatum are thought to encode a "reward prediction error" that is used to train neurons in striatum to reflect the value o a particular action in a particular state. While traditional reinforcement learning models are both simple and effective, they fail to capture at least one striking aspect of human learning behavior: that people learn more from some errors than from others. In particular, people tend to be more influenced by errors so salient as to suggest a context change or ones that occur during a moment of uncertainty. This behavior is well described by abstract statistical models of optimal inference, but the mechanisms by which it could be implemented in the brain remain unknown. Here I examine a potential mechanism by which this rational adjustment of learning might be implemented in the brain: anterior cingulate cortex (ACC), an area of the brain important for behavioral updating, might represent the current context and relay this information to neurons in the striatum encoding action values. By representing a new context after a salient error, ACC may drive the activation of a new set of striatal neurons, thereby discarding the irrelevant information gleaned in the previous context and speeding learning. While such a system allows for rational adjustments in learning, it would require very fine tuned control over the maintenance and discarding of context representations in ACC. One potential mechanism by which this fine tuning might be achieved depends on tonic (persisting) dopamine levels in ACC. Higher tonic dopamine levels are thought to improve network stability which, in ACC, might lead to stable context representations and a rate of learning that is optimized for stable environments. The goal of this proposal is to provide me with training in computational modeling, human EEG measurements, and behavioral pharmacology. This training allows me to test the hypothesis that dopaminergic neuromodulatory systems and networks in ACC serve complementary roles in adjusting influence of outcomes on future actions through two specific Aims. The first Aim will examine whether feedback locked EEG responses emanating from ACC reflect rational adjustments of learning, predict behavioral updating, and are consistent with changes to a context representation. The second Aim will examine whether pharmacologically increasing cortical dopamine levels slows learning and mitigates feedback locked EEG responses.