Reinforcement learning in the human brain: Dimensions, features, and contexts

人类的决策不仅依赖于对世界的逻辑推理，还依赖于试错学习，将特征和行为与奖励和惩罚结果联系起来。这些重复学习机制有着众所周知的神经相关性，这种学习的理论为人类在有限的环境中的行为提供了很好的解释。然而，试错学习通常在实验室任务中进行研究，其中与价值相关的特征是已知的，并且是呈现给受试者的唯一刺激方面。因此，在真实的世界中如何完成这种学习的理论面临着一个关键问题：在任何给定的决定或结果中，都有大量潜在的相关经验，那么应该学习哪些关联并指导未来的行为？这个项目探讨了人类在做出经验指导的决定时如何科普多维世界。对人类决策行为的更好理解将促进更完整的学习理论，使我们能够更好地预测和增强现实世界的决策，并提高对这种决策如何因精神障碍而失败的理解。该项目将为本科生、研究生和博士后学生提供研究机会，并包括以在线演示和解释学习和决策模型的形式进行广泛的公共宣传。此外，拟议的活动包括一个关于MRI方法的年度研讨会，这将提供新方法的培训机会，并向各领域的学生和教师提供外展服务。拟议的项目将使用行为的计算建模和基于模型的fMRI来评估人类如何在奖励引导的决策过程中学习世界的相关和不相关特征。第一项研究将解决是否不相关的刺激尺寸跟踪相对于价值，无论是在选择行为和价值的神经表征。具体来说，该项目将询问腹侧纹状体和大脑其他地方的奖励预测错误信号是否仅由相关的特征-价值关联来解释，或者不相关的特征-价值关联是否也被神经跟踪并影响行为。第二项研究将考察视觉感知领域中习得的统计偶然性是否会任意影响行为和大脑活动方面的价值学习。这项研究的结果将阐明不同的联想学习机制如何相互作用，以指导行为。最后，第三项研究将探讨在奖励导向的决策背景的作用，研究如何以及上下文功能可以纳入决策。这项工作的结果将指导人类强化学习理论的发展，以适应现实世界决策环境的复杂性。这个项目将阐明在学习过程中或在选择时，对哪些特定联想引导行为的控制程度。

Human decision-making depends not only upon logical reasoning about the world, but also trial-and-error learning to associate features and actions with rewarding and punishing outcomes. These reinforcement-learning mechanisms have well-known neural correlates, and theories of such learning provide excellent accounts for human behavior in limited contexts. However, trial-and-error learning has typically been studied in laboratory tasks in which value-associated features are known and are the only stimulus aspects presented to subjects. Thus, theories of how this type of learning is accomplished in the real world face a key problem: there are a multitude of potentially relevant aspects of experience that co-occur with any given decision or outcome, so which associations should be learned and guide future behavior? This project explores how humans cope with a multidimensional world when making experience-guided decisions. A better understanding of human decision-making behavior will facilitate more complete theories of learning, enable us to better predict and enhance real-world decision-making, and improve understanding of how such decision-making might break down due to mental disorders. This project will provide research opportunities for undergraduate, graduate, and postdoctoral students, and include broad public outreach in the form of online demonstrations and explanations of models of learning and decision-making. Further, the proposed activity includes a yearly workshop on MRI methods, which will provide training opportunities in new methods and outreach to students and faculty across fields.The proposed project will use computational modeling of behavior and model-based fMRI to assess how humans learn about relevant and irrelevant features of the world during reward-guided decision-making. The first study will address whether irrelevant stimulus dimensions are tracked with respect to value, both in terms of choice behavior and in terms of neural representations of value. Specifically, the project will ask whether reward prediction error signals in ventral striatum and elsewhere in the brain are explained solely by relevant feature-value associations, or whether irrelevant feature-value associations are also tracked neurally and influence behavior. The second study will examine whether learned statistical contingencies in one domain, visual perception, arbitrarily influence value learning both in terms of behavior and brain activity. Findings from this study will illuminate how distinct associative learning mechanisms interact to guide behavior. Finally, a third study will examine the role of context in reward-guided decision-making, examining how well contextual features can be incorporated into decision-making. The results of this work will guide development of human reinforcement learning theories towards accommodating the complexity of real-world decision-making environments. This project will illuminate the degree to which control over which particular associations guide behavior is exerted during learning or at the time of choice.