Neuron-astrocyte mechanisms of norepinephrine in goal-directed learning

去甲肾上腺素在目标导向学习中的神经元星形胶质细胞机制

• 批准号: 10651486
• 负责人: MRIGANKA SUR
• 金额: $ 64.34万
• 依托单位: MASSACHUSETTS INSTITUTE OF TECHNOLOGY
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-01 至 2028-02-29
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10651486
• 关键词: Affect; Astrocytes; Behavior; Behavioral; Biological; Brain; Brain Diseases; Calcium; Calcium Signaling; Complex; Computer Analysis; Computer Models; Corpus striatum structure; Data; Decision Making; Dependence; Development; Dopamine; Electrophysiology (science); Environment; Equilibrium; Exhibits; Functional disorder; Genetic; Goals; Image; Learning; Linear Models; Measures; Mediating; Memory; Methods; Modeling; Motor Cortex; Mus; Neural Network Simulation; Neuroglia; Neuromodulator; Neurons; Norepinephrine; Outcome; Performance; Phase; Policies; Population; Population Analysis; Population Dynamics; Prefrontal Cortex; Psychological reinforcement; Punishment; Rewards; Role; Signal Transduction; Stimulus; Synapses; System; Task Performances; Uncertainty; Update; Work; behavior measurement; behavioral outcome; cell type; density; designer receptors exclusively activated by designer drugs; dimensional analysis; functional outcomes; genetic manipulation; in vivo; innovation; insight; learned behavior; learning algorithm; locus ceruleus structure; millisecond; neural; neuronal circuitry; neuroregulation; neurotransmission; novel; optogenetics; recurrent neural network; response; spatiotemporal; success; theories; two-photon

Decision-making for goal-directed actions via reinforcement learning (RL) is a fundamental component of complex behaviors. Central to RL theory is the balance between exploration and exploitation, which enables agents to interpret the environment using trial and error to learn an optimal strategy for maximizing reward. Determining the optimal parameters for when to switch between exploration/exploitation states in RL models has been difficult, and thus requires new biological insights. Recent work from our lab implicates locus coeruleus norepinephrine release (LC-NE) in signaling exploration and exploitation states. LC-NE neurons exhibit phasic activity in an RL task when presented with uncertain stimulus evidence to facilitate task execution/exploration, and after receiving a surprising reinforcement to facilitate task optimization/exploitation on the next trial. How these different phasic LC-NE signals are integrated in target regions to modulate different aspects of behavior is unknown. One possibility is through spatiotemporal integration by astrocytes, which are highly responsive to NE, are known to be involved in learning and memory, and can modulate neuronal activity on within-trial and between-trial timescales. Here, we propose that LC-NE release during an RL task causes changes in cortical network dynamics, facilitated through astrocyte signaling, that enable task execution and optimization. We will examine the effects of LC-NE and astrocytes on neuronal population dynamics and RL using innovative approaches combining dual 2-photon imaging of astrocytes and neurons in frontal/prefrontal cortex, high density neural recordings, optogenetic and chemogenetic manipulation of neurons and astrocytes, and computational approaches to define the effects of LC-NE and astrocytes on neuronal populations and task encoding. Finally, we will develop biologically informed computational models of astrocyte-neuron interactions during learned behavior. In Aim 1, we will record cortical astrocytes and neurons in mice performing our RL task. We will use high density single-unit recordings and population analyses to determine how population dynamics evolve during different task epochs. Using this information, we will determine how silencing LC-NE affects astrocyte and neuron computations and dynamics during RL. In Aim 2, we will use chemogenetic and optogenetic manipulations of astrocyte calcium to determine how astrocyte dynamics contribute to RL behaviors, and how this activity affects neuronal population dynamics. In Aim 3, we will examine the hypothesis that extending RL algorithms via NE- astrocyte signals can explain exploration at low stimulus evidence, and that NE-astrocyte interactions across trials would be reflected in policy gradient learning rules to promote exploitation. Finally, we will determine whether incorporating NE-astrocyte-neuron interactions into a recurrent neural network model can provide a rich model for behavior and identify circuit motifs critical to our observed behavioral outcomes. These data will provide an unprecedented view of the role of NE and astrocytes in a crucial behavioral function, and point to ways by which their dysfunction can be ameliorated in brain disorders and diseases.

通过强化学习 (RL) 制定目标导向行动的决策是 复杂的行为。强化学习理论的核心是探索和利用之间的平衡，这使得 代理通过反复试验来解释环境，以学习最大化奖励的最佳策略。 确定 RL 模型中何时在探索/利用状态之间切换的最佳参数 很难，因此需要新的生物学见解。我们实验室最近的工作涉及蓝斑 信号探索和利用状态下的去甲肾上腺素释放（LC-NE）。 LC-NE神经元表现出阶段性 当呈现不确定的刺激证据以促进任务执行/探索时，强化学习任务中的活动， 在收到令人惊讶的强化以促进下一次试验中的任务优化/利用之后。如何 这些不同相位的 LC-NE 信号被整合到目标区域以调节行为的不同方面 未知。一种可能性是通过星形胶质细胞的时空整合，星形胶质细胞对 NE 高度敏感， 已知参与学习和记忆，并且可以调节试验内和记忆中的神经元活动 试验之间的时间尺度。在这里，我们建议 RL 任务期间 LC-NE 的释放会导致皮质的变化 通过星形胶质细胞信号传导促进网络动态，从而实现任务执行和优化。我们将 使用创新方法检查 LC-NE 和星形胶质细胞对神经元群体动态和 RL 的影响 结合额叶/前额叶皮层星形胶质细胞和神经元双 2 光子成像的方法，高密度 神经记录、神经元和星形胶质细胞的光遗传学和化学遗传学操作以及计算 定义 LC-NE 和星形胶质细胞对神经元群体和任务编码影响的方法。最后， 我们将开发学习过程中星形胶质细胞-神经元相互作用的生物信息计算模型 行为。在目标 1 中，我们将记录执行 RL 任务的小鼠的皮质星形胶质细胞和神经元。我们将使用 高密度单单位记录和种群分析，以确定种群动态如何演变 不同的任务时期。利用这些信息，我们将确定沉默 LC-NE 如何影响星形胶质细胞和神经元 强化学习期间的计算和动力学。在目标 2 中，我们将使用化学遗传学和光遗传学操作 星形胶质细胞钙以确定星形胶质细胞动力学如何影响 RL 行为，以及这种活动如何影响 神经元群体动态。在目标 3 中，我们将检验通过 NE 扩展 RL 算法的假设 星形胶质细胞信号可以解释低刺激证据下的探索，并且 NE-星形胶质细胞的相互作用 试验将反映在促进剥削的政策梯度学习规则中。最后，我们将确定 将NE-星形胶质细胞-神经元相互作用纳入循环神经网络模型是否可以提供丰富的 行为模型并识别对我们观察到的行为结果至关重要的电路图案。这些数据将提供 对 NE 和星形胶质细胞在关键行为功能中的作用提出了前所未有的看法，并指出了方法 它们的功能障碍可以在大脑紊乱和疾病中得到改善。