Does prefrontal dopamine modulate error signals to optimally adjust learning?

描述(申请人提供)：人类和动物学习根据过去的经验有效地选择动作。强化学习的一种特殊形式，包括从预测回报的错误中学习，为广泛的学习现象提供了简明的解释。这些模型也为这一过程中涉及的生物机制提供了一些见解。投射到纹状体的多巴胺神经元被认为编码了一种“奖赏预测误差”，用于训练纹状体中的神经元，以反映特定状态下的特定行为的价值。而传统的强化学习模型既是 它们简单而有效，至少未能捕捉到人类学习行为的一个显著方面： 人们从一些错误中学到的东西比从其他错误中学到的更多。特别是，人们往往更容易受到错误的影响，这些错误非常突出，以至于暗示着上下文发生了变化，或者发生在不确定的时刻。最优推理的抽象统计模型很好地描述了这种行为，但它在大脑中实施的机制仍不清楚。在这里，我研究了一种可能的机制，通过它可以在大脑中实现这种合理的学习调整：大脑的前扣带回皮质(ACC)，一个对行为更新很重要的区域，可能代表着当前的背景，并将这一信息传递给纹状体中编码动作值的神经元。通过在显著错误后呈现新的背景，ACC可能会驱动一组新的纹状体神经元的激活，从而丢弃在先前背景中收集的不相关信息，并加速学习。虽然这样的系统允许在学习中进行合理的调整，但它需要对ACC中上下文表示的维护和丢弃进行非常精细的控制。实现这种微调的一个潜在机制取决于ACC中的强效(持续)多巴胺水平。更高的紧张性多巴胺水平被认为可以改善网络稳定性，在ACC中，这可能会导致稳定的背景表征和为稳定环境优化的学习速度。这项提议的目标是为我提供计算建模、人类脑电测量和行为药理学方面的培训。这项训练使我能够检验这一假设，即ACC中的多巴胺能神经调节系统和网络在通过两个特定目标调节结果对未来行动的影响方面起着互补的作用。第一个目标将检查从ACC发出的反馈锁定脑电反应是否反映学习的理性调整，预测行为更新，并与情境表征的变化一致。第二个目标将研究是否从药物上增加皮质多巴胺水平会减缓学习速度，并减轻反馈锁定的脑电反应。

DESCRIPTION (provided by applicant): Humans and animals learn to effectively select actions based on past experience. One particular form of reinforcement learning that involves learning from errors in predicting rewards has provided parsimonious explanations for a broad range of learning phenomena. Such models have also provided some insights into the biological machinery involved in this process. Dopamine neurons projecting to the striatum are thought to encode a "reward prediction error" that is used to train neurons in striatum to reflect the value o a particular action in a particular state. While traditional reinforcement learning models are both simple and effective, they fail to capture at least one striking aspect of human learning behavior: that people learn more from some errors than from others. In particular, people tend to be more influenced by errors so salient as to suggest a context change or ones that occur during a moment of uncertainty. This behavior is well described by abstract statistical models of optimal inference, but the mechanisms by which it could be implemented in the brain remain unknown. Here I examine a potential mechanism by which this rational adjustment of learning might be implemented in the brain: anterior cingulate cortex (ACC), an area of the brain important for behavioral updating, might represent the current context and relay this information to neurons in the striatum encoding action values. By representing a new context after a salient error, ACC may drive the activation of a new set of striatal neurons, thereby discarding the irrelevant information gleaned in the previous context and speeding learning. While such a system allows for rational adjustments in learning, it would require very fine tuned control over the maintenance and discarding of context representations in ACC. One potential mechanism by which this fine tuning might be achieved depends on tonic (persisting) dopamine levels in ACC. Higher tonic dopamine levels are thought to improve network stability which, in ACC, might lead to stable context representations and a rate of learning that is optimized for stable environments. The goal of this proposal is to provide me with training in computational modeling, human EEG measurements, and behavioral pharmacology. This training allows me to test the hypothesis that dopaminergic neuromodulatory systems and networks in ACC serve complementary roles in adjusting influence of outcomes on future actions through two specific Aims. The first Aim will examine whether feedback locked EEG responses emanating from ACC reflect rational adjustments of learning, predict behavioral updating, and are consistent with changes to a context representation. The second Aim will examine whether pharmacologically increasing cortical dopamine levels slows learning and mitigates feedback locked EEG responses.