权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Does prefrontal dopamine modulate error signals to optimally adjust learning?

前额叶多巴胺是否会调节错误信号以最佳地调整学习？

基本信息

批准号：
9142356
负责人：
Matthew Nassar
金额：
$ 2.5万
依托单位：
BROWN UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2014
资助国家：
美国
起止时间：
2014-09-01 至 2017-08-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/9142356
关键词：
Accounting Address Affect Animals Anterior Area Attention deficit hyperactivity disorder Automobile Driving Basal Ganglia Behavior Behavioral Biological Biology Brain Brain region Computer Simulation Corpus striatum structure Data Disease Dopamine Dorsal Electroencephalography Elements Employee Strikes Environment Event Failure Feedback Future Genotype Glean Goals Health Human Lead Learning Learning Module Maintenance Measurement Measures Medial Mediating Mental disorders Modeling N-Methylaspartate Neurons Outcome Pattern Play Prefrontal Cortex Probability Process Protocols documentation Psychological reinforcement Recurrence Rewards Role Schizophrenia Side Signal Transduction Speed Statistical Models Symptoms System Testing Thinking Training Uncertainty Update Ventral Striatum Work abstracting base behavioral pharmacology biophysical model cingulate cortex dopaminergic neuron expectation experience human subject improved insight learned behavior network models neural circuit neuroregulation predicting response research study response tolcapone tool

项目摘要

DESCRIPTION (provided by applicant): Humans and animals learn to effectively select actions based on past experience. One particular form of reinforcement learning that involves learning from errors in predicting rewards has provided parsimonious explanations for a broad range of learning phenomena. Such models have also provided some insights into the biological machinery involved in this process. Dopamine neurons projecting to the striatum are thought to encode a "reward prediction error" that is used to train neurons in striatum to reflect the value o a particular action in a particular state. While traditional reinforcement learning models are both simple and effective, they fail to capture at least one striking aspect of human learning behavior: that people learn more from some errors than from others. In particular, people tend to be more influenced by errors so salient as to suggest a context change or ones that occur during a moment of uncertainty. This behavior is well described by abstract statistical models of optimal inference, but the mechanisms by which it could be implemented in the brain remain unknown. Here I examine a potential mechanism by which this rational adjustment of learning might be implemented in the brain: anterior cingulate cortex (ACC), an area of the brain important for behavioral updating, might represent the current context and relay this information to neurons in the striatum encoding action values. By representing a new context after a salient error, ACC may drive the activation of a new set of striatal neurons, thereby discarding the irrelevant information gleaned in the previous context and speeding learning. While such a system allows for rational adjustments in learning, it would require very fine tuned control over the maintenance and discarding of context representations in ACC. One potential mechanism by which this fine tuning might be achieved depends on tonic (persisting) dopamine levels in ACC. Higher tonic dopamine levels are thought to improve network stability which, in ACC, might lead to stable context representations and a rate of learning that is optimized for stable environments. The goal of this proposal is to provide me with training in computational modeling, human EEG measurements, and behavioral pharmacology. This training allows me to test the hypothesis that dopaminergic neuromodulatory systems and networks in ACC serve complementary roles in adjusting influence of outcomes on future actions through two specific Aims. The first Aim will examine whether feedback locked EEG responses emanating from ACC reflect rational adjustments of learning, predict behavioral updating, and are consistent with changes to a context representation. The second Aim will examine whether pharmacologically increasing cortical dopamine levels slows learning and mitigates feedback locked EEG responses.

描述（由申请人提供）：人类和动物学习根据过去的经验有效地选择行动。强化学习的一种特殊形式涉及从预测奖励的错误中学习，它为广泛的学习现象提供了简洁的解释。这些模型还为该过程中涉及的生物机制提供了一些见解。投射到纹状体的多巴胺神经元被认为编码“奖励预测误差”，用于训练纹状体中的神经元以反映特定状态下特定动作的价值。虽然传统的强化学习模型都是它们简单而有效，但未能捕捉到人类学习行为的至少一个显着方面：人们从某些错误中学到的东西比从其他错误中学到的东西更多。特别是，人们更容易受到错误的影响，这些错误非常明显，以至于表明上下文发生了变化，或者在不确定的时刻发生了错误。这种行为可以通过最佳推理的抽象统计模型得到很好的描述，但在大脑中实现这种行为的机制仍然未知。在这里，我研究了一种可能在大脑中实现这种合理的学习调整的潜在机制：前扣带皮层（ACC）是大脑中对于行为更新很重要的区域，它可能代表当前的环境，并将这些信息传递给纹状体中编码动作值的神经元。通过在显着错误后表示新的上下文，ACC 可以驱动一组新的纹状体神经元的激活，从而丢弃在先前上下文中收集的不相关信息并加速学习。虽然这样的系统允许在学习中进行合理调整，但它需要对 ACC 中上下文表示的维护和丢弃进行非常精细的控制。实现这种微调的一种潜在机制取决于 ACC 中的强效（持续）多巴胺水平。较高的多巴胺水平被认为可以提高网络稳定性，在 ACC 中，这可能会导致稳定的上下文表示和针对稳定环境优化的学习速率。该提案的目标是为我提供计算建模、人类脑电图测量和行为药理学方面的培训。这次训练让我能够检验这样的假设：ACC 中的多巴胺能神经调节系统和网络在通过两个特定目标调整结果对未来行动的影响方面发挥着互补作用。第一个目标将检查来自 ACC 的反馈锁定脑电图反应是否反映了学习的合理调整、预测行为更新以及与上下文表征的变化一致。第二个目标将检查药物增加皮质多巴胺水平是否会减慢学习速度并减轻反馈锁定的脑电图反应。