CRCNS Circuit-Level Mechanisms of Adaptive decision-making

哺乳动物在嘈杂和不稳定的环境中不断适应动作选择的过程，以最大限度地 通过选择可能返回所需结果的行动来成功地做出未来决策 (开采)或在新事物上冒险，看看这是否会产生更好的结果(勘探)。 这种灵活的决策是由皮质-基底节-丘脑(CBGT)回路介导的，这两个回路都控制着 行动选择和使用反馈信号来修改用于未来决策的方法(即，经历 强化学习；RL)。这些通路如何使用反馈来指导未来决策的功能障碍是 许多成瘾行为(如阿片成瘾、肥胖)的主要机制。尽管事实是 决策和RL起源于共同的神经底物，它们通常被研究为 独立的进程。理解动作选择和学习的统一本质需要仔细 重新评估认知算法如何从CBGT网络的电路级动态中出现。 我们提出了一系列的实证和理论研究，这些研究跨越了不同的分析层次，以统一 学习和决策的算法模型，以了解CBGT网络如何使用 反馈以管理勘探和开采之间的权衡。我们为实现这一目标迈出的第一步 目标将是开发一个将认知过程模型连接在一起的计算性“向上映射”框架 在现有行为约束下的CBGT网络的生物现实尖峰模型 从一组适应性决策实验中观察到的结果。这种方法将允许我们推导出 关于不同的CBGT网络属性(例如，种群活动水平或途径)的可测试预测 连接强度)扩展认知过程(例如，证据累积率)以产生不同的 决策政策的表型(具体目标1a)。使用此范例，我们还将生成关于以下内容的预测 在不断变化的条件下，神经可塑性机制如何自适应地将CBGT网络转变为 以适合具体情况的方式管理勘探-开采权衡的不同国家(具体 目标1b)。预测将使用CBGT多个关键地点的录音进行实验测试，以及 双臂强暴啮齿动物纹状体和丘脑底核靶点的光遗传扰动 具有静态或可变行动-结果意外情况的任务(具体目标2)。 相关性(请参阅说明)： 大脑使用反馈来指导未来决策的功能障碍是许多公共健康的主要机制 问题(例如，上瘾、心血管疾病)。这项研究计划将提供新的洞察神经如何 回路引起人类和其他哺乳动物的决策，以及环境背景(例如，波动性)如何 奖励计划)调节大脑网络配置以产生行为灵活性。此信息可以 提供对导致上瘾行为和其他公共健康问题的神经系统的关键见解。

Mammals continuously adapt the process of action selection in noisy and volatile environments to maximize the success of future decisions by either selecting actions that are likely to return a desirable result (exploitation) or taking a risk on something new to see if that will produce a better outcome (exploration). This flexible decision-making is mediated by cortico-basal-ganglia-thalamic (CBGT) circuits that both control action selection and use feedback signals to modify the approach to future decisions (i.e., undergo reinforcement learning; RL). Dysfunction in how these pathways use feedback to guide future decisions is a primary mechanism for many addictive behaviors (e.g., opioid addiction, obesity). Despite the fact that decision-making and RL originate from a common neural substrate, they are generally studied as independent processes. Understanding the unified nature of action selection and learning requires a careful re-evaluation of how cognitive algorithms emerge from the circuit-level dynamics of CBGT networks. We propose a series of empirical and theoretical investigations that bridge across levels of analysis to unify algorithmic models of learning and decision-making in order to understand how CBGT networks use feedback to manage the trade-off between exploration and exploitation. Our first step toward achieving this goal will be to develop a computational “upwards mapping” framework that links cognitive process models with biologically realistic spiking models of CBGT networks under constraints imposed by existing behavioral observations from a set of adaptive decision-making experiments. This approach will allow us to derive testable predictions about how different CBGT network properties (e.g., population activity levels or pathway connection strengths) scale cognitive processes (e.g., evidence accumulation rate) to produce distinct phenotypes of decision policies (Specific Aim 1a). Using this paradigm we will also generate predictions about how, under changing conditions, neural plasticity mechanisms can adaptively shift CBGT networks into distinct states that manage the exploration-exploitation trade-off in contextually appropriate ways (Specific Aim 1b). Predictions will be tested experimentally using recordings in multiple key CBGT sites as well as optogenetic perturbation of striatal and subthalamic nucleus targets in rodents performing a 2-armed bandit task with static or variable action-outcome contingencies (Specific Aim 2). RELEVANCE (See instructions): Dysfunction in how the brain uses feedback to guide future decisions is a primary mechanism for many public health problems (e.g., addiction, cardiovascular disease). This research program will provide new insights into how neural circuits give rise to decision-making in humans and other mammals and how environmental contexts (e.g., volatility of reward schedules) regulate brain network configurations to produce behavioral flexibility. This information can provide key insights into the neural systems that give rise to addictive behaviors and other public health problems.