CRCNS: Neural Basis of Planning

CRCNS：规划的神经基础

基本信息

批准号：
10410402
负责人：
DAEYEOL LEE
金额：
$ 36.9万
依托单位：
JOHNS HOPKINS UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2019
资助国家：
美国
起止时间：
2019-07-02 至 2023-05-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/10410402
关键词：
Algorithms Animals Arbitration Behavior Behavioral Brain Caliber Competence Complex Computers Decision Making Decision Trees Economics Environment Evaluation Exhibits Functional Magnetic Resonance Imaging Future Goals Human Impairment Individual Knowledge Lateral Learning Macaca mulatta Machine Learning Medial Mental Depression Mental disorders Modeling Monitor Monkeys Neurons Neurosciences Outcome Participant Pattern Physiological Play Positioning Attribute Prefrontal Cortex Primates Probability Psyche structure Psychological reinforcement Pupil Research Rewards Role Schizophrenia Series Signal Transduction Speed Strategic Planning Testing Training Trees Update base expectation experimental study fictional works improved learning algorithm neuromechanism neurophysiology nonhuman primate predictive modeling relating to nervous system response simulation source guides

项目摘要

Humans and other animals can choose their actions using multiple learning algorithms and decision making strategies. For example, habitual behaviors adapted to a stable environment can be selected using so-called model-free reinforcement learning algorithms, in which the value of each action is incrementally updated according to the amount of unexpected reward. The underlying neural mechanisms for this type of reinforcement learning have been intensively studied. By contrast, how the brain utilizes the animal's knowledge of its environment to plan sequential actions using a model-based reinforcement learning algorithm remains unexplored. In this application, PIs with complementary expertise will investigate how different subdivisions of the primate prefrontal cortex contribute to the evaluation and arbitration of different learning algorithms during strategic planning in primates, using a sequential game referred to as "4-in-a row". Previous studies have revealed that with training, humans improve their competence in this game by gradually switching away from a model-free reinforcement learning towards a model-based reinforcement learning in the form of a tree search. In the first set of experiments, we will train non-human primates to play the 4-in-a-row game against a computer opponent. We predict that the complexity of the strategic planning and the opponent's move violating the animal's expectation will be reflected in the speed of animal's action and pupil diameters. Next, we will test how the medial and lateral aspects of prefrontal cortex contribute to the evaluation and selection of different learning algorithms during strategic interaction between the animal and computer opponent. We hypothesize that the lateral prefrontal cortex is involved in computing the integrated values of alternative actions originating from multiple sources and guiding the animal's choice, whereas the medial prefrontal cortex might be more involved in monitoring and resolving the discrepancies of actions favored by different learning algorithms. The results from these experiments will expand our knowledge of the neural mechanisms for complex strategic planning and unify various approaches to study naturalistic behaviors. By taking advantage of recent advances in machine learning and decision neuroscience, proposed studies will elucidate how multiple learning algorithms are simultaneously implemented and coordinated via specific patterns of activity in the prefrontal cortex. The results from these studies will transform the behavioral and analytical paradigms used to study high-order planning and their neural underpinnings in humans and animals.

人类和其他动物可以使用多种学习算法和决策策略选择其行动。例如，可以使用所谓的无模型强化学习算法选择适合稳定环境的习惯行为，其中每个动作的价值根据意外奖励的数量逐渐更新。这种增强学习的潜在神经机制已被深入研究。相比之下，大脑如何利用动物对环境的知识来计划基于模型的增强算法的顺序行动，这仍然是出乎意料的。在此应用程序中，具有完整专家的PI将调查灵长类动物前额叶皮层的不同细分如何有助于在初级战略规划过程中使用称为“ 4英寸A行”的战略计划期间的不同学习算法的评估和仲裁。先前的研究表明，通过培训，人类通过逐渐从无模型的强化学习转向以树木搜索的形式转向基于模型的强化学习，从而提高了该游戏的能力。在第一组实验中，我们将训练非人类的素数，以针对计算机选件进行4英寸的比赛。我们预测，战略规划的复杂性和选项的举动违反了动物的期望，将反映在动物的动作速度和瞳孔直径的速度中。接下来，我们将测试在动物和计算机选项之间的战略互动过程中，前额叶皮层的媒体和侧面如何有助于评估和选择不同的学习算法。我们假设横向前额叶皮层参与计算源自多个来源并指导动物选择的替代作用的综合值，而培养基的前额叶皮层可能更多地参与监测和解决不同学习算法所青睐的动作差异和解决动作的差异。这些实验的结果将扩大我们对复杂战略计划的神经元机制的了解，并统一研究自然行为的各种方法。通过利用机器学习和决策神经科学的最新进展，拟议的研究将阐明如何简单地通过前额叶皮层中的特定活动模式实施多个学习算法。这些研究的结果将改变用于研究人类和动物中高阶计划及其神经基础所用的行为和分析范例。