CRCNS Research Proposal: Cortico-amygdalar substrates of adaptive learning

CRCNS 研究提案：适应性学习的皮质杏仁核基质

• 批准号: 10455591
• 负责人: Alicia Izquierdo
• 金额: $ 31.29万
• 依托单位: DARTMOUTH COLLEGE
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-15 至 2024-02-29
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10455591
• 关键词: Activities of Daily Living; Amygdaloid structure; Area; Behavioral; Brain; Brain region; Calcium; Computer Models; Data; Dissection; Dorsal; Environment; Feedback; Food; Functional Magnetic Resonance Imaging; Grain; Human; Image; Learning; Metaplasia; Modeling; Monitor; Monkeys; Neurons; Organism; Outcome; Pathway interactions; Property; Rattus; Research Proposals; Resolution; Reversal Learning; Rewards; Rodent; Role; Schedule; Signal Transduction; Speed; Stimulus; Study models; Synapses; System; Techniques; Testing; Time; Uncertainty; Update; adaptive learning; cell type; experimental study; learning strategy; network models; neuromechanism; predictive modeling; preference; response; statistics; visual stimulus; word learning

PROJECT DESCRIPTION 1. BACKGROUND AND SIGNIFICANCE Learning from feedback in the real w'orld is limited by constant fluctuations in reward outcomes associated with choosing certain options or actions. Some of these fluctuations are caused by fundamental changes in the reward values of those options/actions that necessitate dramatic adjustments to the current learning strategies, like in epiphany learning or one-shot learning [Chen & Krajbich, 2017; Lee et al. 2015]. Other changes represent inherent stochasticity in an otherwise stable environment and should be tolerated and ignored to maintain stable choice preferences. In other words, learning in dynamic environments is bounded by a tradeoff between being adaptable (i.e. respond quickly to changes in the environment) and being precise (i.e. update slowly after each feedback to be more accurate), which we refer to as the adaptability-precision tradeoff [Farashahi et al., 2017; Khorsand & Soltani, 2017]. Therefore, distinguishing meaningful changes in the environment from natural fluctuations can greatly enhance adaptive learning, indicating that adaptive learning depends on interactions between multiple brain areas. To date, most computational models of learning under uncertainty are very high-level and/or descriptive [Behrens et al., 2007; Costa et al., 2015; ligaya, 2016; Jang et al., 2015; Nassar et al., 201 O; Payzan-LeNestour & Bossaerts, 2011] and therefore, do not provide specific testable predictions. On the other hand, neural mechanisms of uncertainty monitoring for adaptive learning have been predominantly investigated in humans, and in a few cases monkeys, both of which are limited in terms of circuit-level manipulations. However, interactions between brain areas unfold on short timescales and can be specific to certain cell types. These properties have severely limited the ability of functional MRI [Logothetis, 2003] or MEG [Dale et al., 2000; Mostert et al., 2015] to reveal the microcircuit mechanisms within brain regions and fine-grained contributions between brain regions. To overcome these limitations and reveal neural mechanisms underlying adaptive learning under uncertainty, we propose a combination of detailed computational modeling, imaging of stable neuronal ensembles, and precise system-level manipulation of interactions between multiple brain areas in rodents. The latter is possible in part due to powerful circuit- dissection techniques in rodents that allow manipulations of genetically-tractable cell types and thus, specific projections between brain regions. Combined with decoding of neuronal activity in cortex and guided by mechanistic computational modeling, this approach enables us to investigate both microcircuit and system-level mechanisms of adaptive learning under uncertainty. We have recently proposed a mechanistic model for adaptive learning under uncertainty [Farashahi et al., 2017]. This model, which we refer to as reward-dependent metaplasticity (ROMP) model, provides a synaptic mechanism for how learning can be self-adjusted to reward statistics in the environment. The model predicts as more time spent in a given environment with a certain reward schedule, the organisms should become less sensitive to feedback that does not support what is learned. This and other predictions of the model were confirmed using a large set of behavioral data in monkeys during a probabilistic reversal learning task [Farashahi et al., 2017]. Although the proposed metaplasticity mechanism enables the model to become more robust against random fluctuations, it also causes the model to not respond quickly to actual changes in the environment. This limitation can be partially mitigated by allowing synapses to become unstable in response to changes in the environment [ligaya, 2016]. Interestingly, in our model, the changes in the activity of neurons that encode reward values can be used by another system to compute volatility in the environment. This signal can be used subsequently to increase the speed of learning when volatility is high, that is, when there is a higher chance of real changes in the environment. We hypothesize that such interactions between value-encoding and uncertainty-monitoring systems can enhance adaptability required in dynamic environments. In addition to this modeling study, we recently have shown that both basolateral amygdala (BLA) and orbitofrontal cortex (OFC) have complementary roles in adaptive value learning under uncertainty in rodents [Stolyarova & Izquierdo, 2017]. In this experiment, rats learned the variance in delays for food rewards associated with different visual stimuli upon selecting between them. We found that OFC is necessary to accurately learn such stimulus-outcome association (in terms of 1 21

项目说明 一、背景和意义 在现实世界中从反馈中学习受到奖励结果不断波动的限制 与选择某些选项或动作相关的。其中一些波动是由 这些选项/行动的奖励价值发生了根本性的变化，这是有必要的 调整当前的学习策略，如顿悟学习或一次性学习[Chen& Krajbich，2017；Lee等人。2015年]。其他更改表示否则中的固有随机性 稳定的环境，应该容忍和忽略，以保持稳定的选择偏好。在……里面 换句话说，在动态环境中学习受到适应性和适应性之间的权衡 (即快速响应环境中的变化)和精确(即每次更新缓慢 反馈更准确)，我们称之为适应性和精确度之间的权衡[Farashahi et Al，2017；霍尔桑德和索尔塔尼，2017]。因此，区分 来自自然波动的环境可以大大增强适应性学习，表明适应性 学习依赖于多个大脑区域之间的相互作用。 到目前为止，大多数不确定性下的学习计算模型都是非常高水平的和/或 描述性[Behrens等人，2007；Costa等人，2015；Ligaya，2016；Jang等人，2015；Nassar等人， 201O；Payzan-LeNestour&Bossaerts，2011]，因此不提供特定的可测试性 预测。另一方面，自适应学习中不确定性监控的神经机制 主要在人类身上进行研究，少数情况下是在猴子身上进行研究，这两种动物都是 在电路级操作方面受到限制。然而，大脑区域之间的相互作用在 时间很短，并且可以特定于某些细胞类型。这些属性严重限制了 功能磁共振成像[Logothetis，2003]或MEG[Dale等人，2000；Mostert等人，2015]揭示 脑区内的微电路机制和脑之间的细粒贡献 地区。为了克服这些限制并揭示适应的神经机制 在不确定情况下的学习，我们提出了一种详细的计算模型， 稳定的神经元群的成像和相互作用的精确系统级操作 在啮齿动物的多个脑区之间。后者之所以可能，部分原因是强大的电路- 啮齿类动物的解剖技术，允许操纵遗传上易驯化的细胞类型，因此， 大脑区域之间的特定投射。结合大脑皮层神经元活动的解码 在机械计算模型的指导下，这种方法使我们能够研究 不确定条件下自适应学习的微电路和系统级机制。 最近，我们提出了一种不确定条件下的自适应学习机制模型 [Farashahi等人，2017年]。这个模型，我们称之为报酬依赖的元可塑性(ROMP) 模型，为学习如何自我调整以奖励统计数据提供了一种突触机制 环境。该模型预测，在给定的环境中花费更多的时间 奖励计划，有机体应该对不支持的反馈变得不那么敏感 学到了什么。该模型的这一预测和其他预测使用了一大组 猴子在概率逆转学习任务中的行为数据[Farashahi等人，2017]。 尽管所提出的超塑性机制能够使模型变得更健壮 随机波动，它还会导致模型不能快速响应 环境。允许突触在脑内变得不稳定可以部分缓解这种限制 对环境变化的反应[Ligaya，2016]。有趣的是，在我们的模型中， 编码奖励值的神经元的活动可以被另一个系统用来计算 环境的波动性。随后可以使用该信号来提高学习速度 当波动性很高时，也就是当环境发生真正变化的可能性更高时。我们 假设价值编码和不确定性监测系统之间的这种相互作用可以 增强在动态环境中所需的适应性。 除了这项模型化研究，我们最近还发现，杏仁基底外侧核 在适应性价值学习中，大脑皮质(BLA)和眶前叶皮质(OFC)具有互补作用 啮齿动物的不确定性[Stolyarova&Izquierdo，2017]。在这项实验中，大鼠学习了 在不同的视觉刺激之间进行选择时，食物奖励的延迟。我们 发现OFC对于准确地学习这种刺激-结果关联是必要的(就 1 21岁