CRCNS Circuit-Level Mechanisms of Adaptive decision-making

CRCNS 自适应决策的电路级机制

• 批准号: 10261528
• 负责人: TIMOTHY D VERSTYNEN
• 金额: $ 33.68万
• 依托单位: CARNEGIE-MELLON UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-30 至 2023-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10261528
• 关键词: Addictive Behavior; Algorithms; Animals; Basal Ganglia; Behavior; Behavioral; Biological; Brain; Cardiovascular Diseases; Cognitive; Conflict (Psychology); Corpus striatum structure; Decision Making; Environment; Equilibrium; Evaluation; Feedback; Frequencies; Functional disorder; Future; Goals; Human; Instruction; Investigation; Learning; Link; Mammals; Maps; Mediating; Modeling; Nature; Neuronal Plasticity; Obesity; Opiate Addiction; Outcome; Pathway interactions; Phenotype; Policies; Population; Probability; Process; Property; Psychological reinforcement; Public Health; Research; Rewards; Risk; Rodent; Schedule; Series; Signal Transduction; Site; Structure of subthalamic nucleus; System; Testing; Thalamic structure; addiction; behavior observation; cell type; cognitive process; computer framework; density; experimental study; flexibility; in vivo; insight; neural circuit; neural network; novel strategies; optogenetics; programs; relating to nervous system; success

Mammals continuously adapt the process of action selection in noisy and volatile environments to maximize the success of future decisions by either selecting actions that are likely to return a desirable result (exploitation) or taking a risk on something new to see if that will produce a better outcome (exploration). This flexible decision-making is mediated by cortico-basal-ganglia-thalamic (CBGT) circuits that both control action selection and use feedback signals to modify the approach to future decisions (i.e., undergo reinforcement learning; RL). Dysfunction in how these pathways use feedback to guide future decisions is a primary mechanism for many addictive behaviors (e.g., opioid addiction, obesity). Despite the fact that decision-making and RL originate from a common neural substrate, they are generally studied as independent processes. Understanding the unified nature of action selection and learning requires a careful re-evaluation of how cognitive algorithms emerge from the circuit-level dynamics of CBGT networks. We propose a series of empirical and theoretical investigations that bridge across levels of analysis to unify algorithmic models of learning and decision-making in order to understand how CBGT networks use feedback to manage the trade-off between exploration and exploitation. Our first step toward achieving this goal will be to develop a computational “upwards mapping” framework that links cognitive process models with biologically realistic spiking models of CBGT networks under constraints imposed by existing behavioral observations from a set of adaptive decision-making experiments. This approach will allow us to derive testable predictions about how different CBGT network properties (e.g., population activity levels or pathway connection strengths) scale cognitive processes (e.g., evidence accumulation rate) to produce distinct phenotypes of decision policies (Specific Aim 1a). Using this paradigm we will also generate predictions about how, under changing conditions, neural plasticity mechanisms can adaptively shift CBGT networks into distinct states that manage the exploration-exploitation trade-off in contextually appropriate ways (Specific Aim 1b). Predictions will be tested experimentally using recordings in multiple key CBGT sites as well as optogenetic perturbation of striatal and subthalamic nucleus targets in rodents performing a 2-armed bandit task with static or variable action-outcome contingencies (Specific Aim 2). RELEVANCE (See instructions): Dysfunction in how the brain uses feedback to guide future decisions is a primary mechanism for many public health problems (e.g., addiction, cardiovascular disease). This research program will provide new insights into how neural circuits give rise to decision-making in humans and other mammals and how environmental contexts (e.g., volatility of reward schedules) regulate brain network configurations to produce behavioral flexibility. This information can provide key insights into the neural systems that give rise to addictive behaviors and other public health problems.

哺乳动物在噪声和挥发性环境中不断调整动作选择过程以最大化 通过选择可能返回理想结果的动作，未来决策的成功 （剥削）或冒险冒险新事物，看看这是否会产生更好的结果（探索）。 这种灵活的决策由Cortico-Basal-Ganglia-Thalamic（CBGT）循环介导 行动选择并使用反馈信号来修改未来决策的方法（即进行 强化学习； RL）。这些途径如何使用反馈指导未来决策的功能障碍是 许多其他行为的主要机制（例如阿片类药物成瘾，肥胖）。尽管事实 决策和RL起源于常见的神经底物，它们通常被研究为 独立过程。了解行动选择和学习的统一性质需要仔细 重新评估认知算法如何从CBGT网络的电路级动力学中出现。 我们提出了一系列经验和理论研究，这些研究跨越了分析水平，以统一 学习和决策的算法模型，以了解CBGT网络的使用方式 反馈以管理勘探和剥削之间的权衡。我们迈向实现这一目标的第一步 目标是开发一个链接认知过程模型的计算“向上映射”框架 CBGT网络的生物学上现实的尖峰模型受到现有行为施加的约束 一组自适应决策实验的观察。这种方法将使我们得出 关于不同CBGT网络属性的可检验预测（例如，人口活动水平或途径 连接强度）规模认知过程（例如，证据积累率）产生不同的 决策政策的表型（特定目标1A）。使用此范式，我们还将生成有关的预测 在不断变化的条件下，神经可塑性机制如何适应地将CBGT网络转移到 以上下文适当的方式管理勘探探索折衷的不同状态（具体 目标1b）。将使用多个关键CBGT站点中的记录以及 啮齿动物的纹状体和丘脑下核靶标的光遗传学扰动 具有静态或可变动作结果突发事件的任务（特定目标2）。 相关性（请参阅说明）： 大脑如何使用反馈指导未来决策的功能障碍是许多公共卫生的主要机制 问题（例如成瘾，心血管疾病）。该研究计划将提供有关中立如何中立的新见解 电路引起人类和其他哺乳动物的决策以及环境环境如何（例如，波动性 奖励时间表）调节大脑网络配置以产生行为灵活性。此信息可以 对神经元系统提供关键见解，从而引起其他行为和其他公共卫生问题。