The neural computations supporting hierarchical reinforcement learning

支持分层强化学习的神经计算

基本信息

  • 批准号:
    10359201
  • 负责人:
  • 金额:
    $ 38.01万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
  • 财政年份:
    2019
  • 资助国家:
    美国
  • 起止时间:
    2019-04-01 至 2024-01-31
  • 项目状态:
    已结题

项目摘要

The neural computations supporting hierarchical reinforcement learning - Project Summary. This project explores how humans learn at multiple hierarchical levels in parallel, and how this supports human intelligence. Human decisions are typically hierarchically structured: we make high-level decisions (making a cup of coffee), which constrain lower level decisions (grinding coffee beans, boiling water, etc.), which themselves constrain simpler and simpler decisions and motor actions. This hierarchy in decisions is paralleled by a hierarchy in our representation of our environment: some sensory signals trigger simple decisions (a red light signals a stop), while other signal a broader, more abstract behavioral change (rain signals a set of adaptations when driving). Thus, complex hierarchical structure underlies the way we respond to our environment in seemingly simple, everyday tasks. This ability is supported by the prefrontal cortex, which represents states and decisions at multiple degrees of hierarchical abstraction. My previous work shows that hierarchical representations support transfer and generalization while learning, an ability that artificial agents still struggle to match human performance in. However, how we learn to form these hierarchical representations is poorly understood, despite how crucial it is for human intelligence. The proposed work will examine how multiple, parallel hierarchical loops between prefrontal cortex and the basal ganglia support reinforcement learning at multiple hierarchical levels in parallel, and how this promotes flexible behavior. To this end, we will address three aims: 1. We will show that the same reinforcement learning computations happen in parallel at multiple levels of abstraction, as hypothesized by our computational model of prefrontal- subcortical networks. 2. We will demonstrate that humans partition learning problems into multiple sequential subgoals so they can learn multiple simple strategies instead of one complex strategy, and that reusing these simple strategies promotes fast exploration and learning. 3. We will show that hierarchical learning does not rely exclusively on rewards, but that novelty signals are crucial for identifying subgoals and learning through curiosity. Across all three aims, we will use behavioral experiments in conjunction with computational modeling to characterize how humans learn hierarchically. In addition, we will use EEG and fMRI to identify the neural computations underlying the cognitive systems inferred from behavior and modeling. This project will provide new insights into the computational mechanisms that give rise to learning, and thus provide a better handle on the sources of learning dysfunction observed in many psychiatric diseases, including schizophrenia, depression, anxiety, ADHD, and OCD. Additionally, it will provide new tools, in the form of experimental protocols and precise computational models, for studying learning across populations and species.
支持分层强化学习的神经计算--项目总结。 这个项目探索了人类如何并行地在多个层次上学习,以及这如何支持人类 智慧。人类的决策通常是有层次结构的:我们做出高级别的决策(做出 咖啡),这限制了较低级别的决策(研磨咖啡豆、沸水等),这些决定 它们本身制约着越来越简单的决定和运动动作。决策中的这种等级是平行的 根据我们对环境的表示中的层次结构:一些感觉信号会触发简单的决定(红色 光表示停止),而其他表示更广泛、更抽象的行为变化(雨表示一组 驾驶时的适应)。因此,复杂的等级结构是我们回应我们的 在看似简单的日常任务中创造良好的环境。这种能力由前额叶皮质支持,前额叶皮质 表示多个层次抽象程度的状态和决策。我之前的工作表明, 层次表示支持边学习边迁移和泛化,这是人工智能体 仍在努力与人类的表现相媲美。然而,我们如何学习形成这些等级 尽管陈述对人类智力有多么重要,但人们对它的理解很少。拟议的工作将 研究前额叶皮质和基底神经节之间的多个平行层次环路如何支持 多个层级并行的强化学习,以及这如何促进灵活行为。至 为此,我们将解决三个目标:1.我们将证明相同的强化学习计算 在多个抽象层次上并行发生,正如我们的前额叶计算模型所假设的那样- 皮质下网络。2.我们将演示人类将学习问题划分为多个顺序 子目标,这样他们就可以学习多个简单的策略,而不是一个复杂的策略,并重复使用这些策略 简单的策略有助于快速探索和学习。3.我们将证明分层学习不会 完全依赖于奖励,但新奇的信号对于确定子目标和学习至关重要 好奇心。在所有三个目标中,我们将结合使用行为实验和计算建模 来刻画人类如何层次分明地学习。此外,我们将使用脑电和功能磁共振来识别神经 从行为和建模中推断出的认知系统的潜在计算。该项目将提供 对促进学习的计算机制的新见解,从而提供了更好的处理 在包括精神分裂症在内的许多精神疾病中观察到的学习障碍的来源, 抑郁、焦虑、多动症和强迫症。此外,它将提供新的工具,以试验性的形式 协议和精确的计算模型,用于研究跨种群和物种的学习。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Anne G.E. Collins其他文献

Dual effects of dual-tasking on instrumental learning
  • DOI:
    10.1016/j.cognition.2025.106228
  • 发表时间:
    2025-11-01
  • 期刊:
  • 影响因子:
    2.800
  • 作者:
    Huang Ham;Samuel D. McDougle;Anne G.E. Collins
  • 通讯作者:
    Anne G.E. Collins
A goal-centric outlook on learning
以目标为中心的学习观
  • DOI:
    10.1016/j.tics.2023.08.011
  • 发表时间:
    2023-12-01
  • 期刊:
  • 影响因子:
    17.200
  • 作者:
    Gaia Molinaro;Anne G.E. Collins
  • 通讯作者:
    Anne G.E. Collins

Anne G.E. Collins的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Anne G.E. Collins', 18)}}的其他基金

Thalamocortical cognitive networks in the healthy human brain
健康人脑中的丘脑皮质认知网络
  • 批准号:
    10633809
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Developing artificial neural network tools for cognitive modeling
开发用于认知建模的人工神经网络工具
  • 批准号:
    10641215
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
  • 批准号:
    10113371
  • 财政年份:
    2019
  • 资助金额:
    $ 38.01万
  • 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
  • 批准号:
    10576384
  • 财政年份:
    2019
  • 资助金额:
    $ 38.01万
  • 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
  • 批准号:
    9894854
  • 财政年份:
    2019
  • 资助金额:
    $ 38.01万
  • 项目类别:

相似海外基金

Impact of tissue resident memory T cells on the neuro-immune pathophysiology of anterior eye disease
组织驻留记忆 T 细胞对前眼疾病神经免疫病理生理学的影响
  • 批准号:
    10556857
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Fear and anxiety circuit mechanisms in anterior hypothalamic nucleus
下丘脑前核的恐惧和焦虑环路机制
  • 批准号:
    10789153
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Elucidating signaling networks in Anterior Segment development, repair and diseases
阐明眼前节发育、修复和疾病中的信号网络
  • 批准号:
    10718122
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
The Intimate Interplay Between Keratoconus, Sex Hormones, and the Anterior Pituitary
圆锥角膜、性激素和垂体前叶之间的密切相互作用
  • 批准号:
    10746247
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Anterior Insula Projections for Alcohol Drinking/Anxiety Interactions in Female and Male Rats
雌性和雄性大鼠饮酒/焦虑相互作用的前岛叶预测
  • 批准号:
    10608759
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Impact of tissue resident memory T cells on the neuro-immunepathophysiology of anterior eye disease
组织驻留记忆 T 细胞对前眼疾病神经免疫病理生理学的影响
  • 批准号:
    10804810
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Investigation of the effect of anterior eye shape on myopia progression due to prolonged near work.
研究因长时间近距离工作而导致的前眼形状对近视进展的影响。
  • 批准号:
    23K09063
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Generation and characterization of anterior pituitary stem cells from human pluripotent stem cells
人多能干细胞垂体前叶干细胞的产生和表征
  • 批准号:
    23K08005
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Anterior cruciate ligament injury: towards a gendered environmental approach
前十字韧带损伤:走向性别环境方法
  • 批准号:
    485090
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Operating Grants
EASI-TOC: Endovascular Acute Stroke Intervention-Tandem OCclusion: atrial of acute cervical internal carotid artery stenting during endovascularthrombectomy for anterior circulation stroke
EASI-TOC:血管内急性卒中干预-串联闭塞:前循环卒中血管内血栓切除术期间急性颈内动脉心房支架置入术
  • 批准号:
    490056
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Operating Grants
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了