Orbitofrontal mediation of competitive cue interactions in reward learning

奖励预测线索通过驱动渴望和寻求奖励在物质使用障碍（SUD）中发挥着关键作用 尽管有不良后果，但仍然存在行为，这使得了解预测信用如何分配至关重要。 目前的共识是，线索奖励学习取决于预测误差（PE）的计算， 多巴胺能系统的教学信号，驱动下游的联想表征的更新 地区因此，如何分配预测信用的问题可以归结为如何计算PE。早期 学习模型假设每个线索产生自己独立的PE，这意味着信用分配只是 这取决于每个线索与奖励的相关性，而与其他线索无关。然而，自从发现线索竞争现象以来，主流观点一直认为奖励学习是由一个单一的， 所有线索之间共享的总PE，因此，只有最好的奖励预测者才会获得实质性的回报。 信用和竞争优势不太相关的线索。关键的是，有证据表明， 非竞争性学习可能是个体差异的学习风格谱的两端， 对SUD的影响。我们认为这些差异可能构成SUD的风险因素，因为 竞争性学习是，更多的信贷将分配给主机的线索，否则会被淘汰。 这个研究项目的首要目标是阐明决定个人行为的神经机制。 学习风格作为这一方向的第一步，该建议侧重于眶额皮层的作用， 这与奖励处理的各个方面有关。我们之前假设OFC作为 一个主要的枢纽，在这里，由多个线索引起的奖励预期可以相加，形成一个集合 期望值-计算总PE的关键步骤。因此，这一假说的一个明确预测是，OFC 应该在竞争性学习中发挥关键作用。本提案中的实验将使用 OFC中的体内电生理学（目的1和2）和光遗传学刺激（目的3）与一种 一项新的任务，专门设计来研究学习风格及其神经基质的个体差异。 这项任务的独特之处在于，它将竞争性和非竞争性的学习风格相互冲突。 使用该任务的初步结果证实了个体差异与以下程度相关： 眶额皮层的神经活动根据竞争性和非竞争性风格对奖赏预期进行编码。 有趣的是，这两种风格都是在学习过程中编码的，但在提示时期内表现出不同的时间动态， 这表明了两个相对独立的学习系统之间的相互作用。此外，我们的试点数据表明， OFC在竞争性学习中的因果作用。因此，这项建议的结果将进一步加深我们对以下问题的理解： 奖励学习机制，这对于开发SUD的新治疗策略至关重要。

Reward-predictive cues play a key role in substance use disorders (SUDs) by driving craving and reward-seeking behaviors despite adverse consequences, making it essential to understand how predictive credit is assigned. The current consensus is that cue-reward learning depends on the computation of prediction error (PEs) by the dopaminergic system—the teaching signal that drives the updating of associative representations in downstream areas. The question of how predictive credit is assigned may thus come down to how PE are computed. Early learning models assumed that each cue generates its own separate PE, implying that credit assignment is simply determined by each cue’s correlation with reward independently of other cues. Since the discovery of cue-competition phenomena, however, the dominant view has been that reward learning is driven by a single, aggregate PE shared among all cues, as a result of which only the best reward predictors will acquire substantial credit and outcompete less well correlated cues. Criticaly, converging evidence suggests that competitive and noncompetitive learning might be the two ends of a learning-style spectrum of individual differences, with deep implications for SUDs. We propose that these differences may constitute a risk factor for SUDs because the less competitive learning is, the more credit will be assigned to as host of cues that would otherwise get outcompeted. The overarching goal of this research project is to elucidate the neural mechanisms that determine an individual’s learning style. As a first step in this direction, this proposal focuses on the role of the orbitofrontal cortex, a region that has been implicated in various aspects of reward processing. We previously hypothesized that OFC acts as a major hub where reward expectancies elicited by multiple cues can be summed to form an aggregate expectancy—the key step to computing an aggregate PE. Thus, a clear prediction of this hypothesis is that OFC should play a critical role in competitive learning. The experiments in this proposal will test this prediction using in-vivo electrophysiological (Aims 1 and 2) and optogenetic stimulation (Aim 3) in OFC in combination with a novel task specifically designed to examine individual differences in learning style and their neural substrates. This task is unique in that it sets competitive and noncompetitive learning styles in conflict with one another. Preliminary results using this task has confirmed individual differences that correlate with the extent to which neural activity in OFC encodes reward expectancies based on competitive vs. noncompetitive styles. Interestingly, both styles are encoded during learning but show different temporal dynamics within the cue epoch, suggesting the interplay between two relatively independent learning systems. In addition, our pilot data indicates a causal role of OFC in competitive learning. Findings from this proposal will thus further our understanding of reward learning mechanism, which is essential for developing new therapeutic strategies for SUDs.