CRCNS Circuit-Level Mechanisms of Adaptive decision-making

哺乳动物在嘈杂和不稳定的环境中不断适应行动选择的过程， 通过选择可能返回期望结果的行动来实现未来决策的成功 （开发）或在新事物上冒险，看看是否会产生更好的结果（探索）。 这种灵活的决策是由皮质-基底神经节-丘脑（CBGT）回路介导的， 动作选择并使用反馈信号来修改将来决策的方法（即，经历 强化学习（RL）。这些途径如何使用反馈来指导未来决策的功能障碍是一个 许多成瘾行为的主要机制（例如，阿片类药物成瘾、肥胖）。尽管事实上 决策和强化学习起源于一个共同的神经基质，它们通常被研究为 独立的进程。要理解动作选择和学习的统一本质， 重新评估认知算法如何从CBGT网络的电路级动态中出现。 我们提出了一系列的实证和理论研究，跨越分析水平，以统一 学习和决策的算法模型，以了解CBGT网络如何使用 反馈，以管理探索和开发之间的权衡。我们实现这一目标的第一步 我的目标是开发一个计算的“向上映射”框架，将认知过程模型联系起来 CBGT网络的生物现实的尖峰模型，在现有的行为约束下， 从一组自适应决策实验的观察。这种方法将使我们能够获得 关于不同CBGT网络属性（例如，种群活动水平或途径 连接强度）衡量认知过程（例如，证据积累率）， 决策政策的表型（具体目标1a）。使用这个范例，我们还将生成关于 在不断变化的条件下，神经可塑性机制如何自适应地将CBGT网络转变为 不同的国家，管理的勘探开发权衡在上下文适当的方式（具体 目标1b）。预测将使用多个关键CBGT站点的记录以及 啮齿类动物执行双臂强盗行为时纹状体和丘脑底核靶点的光遗传学扰动 具有静态或可变行动结果偶然性的任务（具体目标2）。 相关性（参见说明）： 大脑如何使用反馈来指导未来决策的功能障碍是许多公共卫生的主要机制 问题（例如，成瘾、心血管疾病）。这项研究计划将提供新的见解如何神经 电路引起人类和其他哺乳动物的决策以及环境背景（例如，波动 奖励时间表）调节大脑网络配置以产生行为灵活性。该信息可以 提供关键的见解，神经系统引起成瘾行为和其他公共卫生问题。

Mammals continuously adapt the process of action selection in noisy and volatile environments to maximize the success of future decisions by either selecting actions that are likely to return a desirable result (exploitation) or taking a risk on something new to see if that will produce a better outcome (exploration). This flexible decision-making is mediated by cortico-basal-ganglia-thalamic (CBGT) circuits that both control action selection and use feedback signals to modify the approach to future decisions (i.e., undergo reinforcement learning; RL). Dysfunction in how these pathways use feedback to guide future decisions is a primary mechanism for many addictive behaviors (e.g., opioid addiction, obesity). Despite the fact that decision-making and RL originate from a common neural substrate, they are generally studied as independent processes. Understanding the unified nature of action selection and learning requires a careful re-evaluation of how cognitive algorithms emerge from the circuit-level dynamics of CBGT networks. We propose a series of empirical and theoretical investigations that bridge across levels of analysis to unify algorithmic models of learning and decision-making in order to understand how CBGT networks use feedback to manage the trade-off between exploration and exploitation. Our first step toward achieving this goal will be to develop a computational “upwards mapping” framework that links cognitive process models with biologically realistic spiking models of CBGT networks under constraints imposed by existing behavioral observations from a set of adaptive decision-making experiments. This approach will allow us to derive testable predictions about how different CBGT network properties (e.g., population activity levels or pathway connection strengths) scale cognitive processes (e.g., evidence accumulation rate) to produce distinct phenotypes of decision policies (Specific Aim 1a). Using this paradigm we will also generate predictions about how, under changing conditions, neural plasticity mechanisms can adaptively shift CBGT networks into distinct states that manage the exploration-exploitation trade-off in contextually appropriate ways (Specific Aim 1b). Predictions will be tested experimentally using recordings in multiple key CBGT sites as well as optogenetic perturbation of striatal and subthalamic nucleus targets in rodents performing a 2-armed bandit task with static or variable action-outcome contingencies (Specific Aim 2). RELEVANCE (See instructions): Dysfunction in how the brain uses feedback to guide future decisions is a primary mechanism for many public health problems (e.g., addiction, cardiovascular disease). This research program will provide new insights into how neural circuits give rise to decision-making in humans and other mammals and how environmental contexts (e.g., volatility of reward schedules) regulate brain network configurations to produce behavioral flexibility. This information can provide key insights into the neural systems that give rise to addictive behaviors and other public health problems.