The neural computations supporting hierarchical reinforcement learning

支持分层强化学习的神经计算

基本信息

  • 批准号:
    10359201
  • 负责人:
  • 金额:
    $ 38.01万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
  • 财政年份:
    2019
  • 资助国家:
    美国
  • 起止时间:
    2019-04-01 至 2024-01-31
  • 项目状态:
    已结题

项目摘要

The neural computations supporting hierarchical reinforcement learning - Project Summary. This project explores how humans learn at multiple hierarchical levels in parallel, and how this supports human intelligence. Human decisions are typically hierarchically structured: we make high-level decisions (making a cup of coffee), which constrain lower level decisions (grinding coffee beans, boiling water, etc.), which themselves constrain simpler and simpler decisions and motor actions. This hierarchy in decisions is paralleled by a hierarchy in our representation of our environment: some sensory signals trigger simple decisions (a red light signals a stop), while other signal a broader, more abstract behavioral change (rain signals a set of adaptations when driving). Thus, complex hierarchical structure underlies the way we respond to our environment in seemingly simple, everyday tasks. This ability is supported by the prefrontal cortex, which represents states and decisions at multiple degrees of hierarchical abstraction. My previous work shows that hierarchical representations support transfer and generalization while learning, an ability that artificial agents still struggle to match human performance in. However, how we learn to form these hierarchical representations is poorly understood, despite how crucial it is for human intelligence. The proposed work will examine how multiple, parallel hierarchical loops between prefrontal cortex and the basal ganglia support reinforcement learning at multiple hierarchical levels in parallel, and how this promotes flexible behavior. To this end, we will address three aims: 1. We will show that the same reinforcement learning computations happen in parallel at multiple levels of abstraction, as hypothesized by our computational model of prefrontal- subcortical networks. 2. We will demonstrate that humans partition learning problems into multiple sequential subgoals so they can learn multiple simple strategies instead of one complex strategy, and that reusing these simple strategies promotes fast exploration and learning. 3. We will show that hierarchical learning does not rely exclusively on rewards, but that novelty signals are crucial for identifying subgoals and learning through curiosity. Across all three aims, we will use behavioral experiments in conjunction with computational modeling to characterize how humans learn hierarchically. In addition, we will use EEG and fMRI to identify the neural computations underlying the cognitive systems inferred from behavior and modeling. This project will provide new insights into the computational mechanisms that give rise to learning, and thus provide a better handle on the sources of learning dysfunction observed in many psychiatric diseases, including schizophrenia, depression, anxiety, ADHD, and OCD. Additionally, it will provide new tools, in the form of experimental protocols and precise computational models, for studying learning across populations and species.
支持分层强化学习的神经计算-项目摘要。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Anne G.E. Collins其他文献

Dual effects of dual-tasking on instrumental learning
  • DOI:
    10.1016/j.cognition.2025.106228
  • 发表时间:
    2025-11-01
  • 期刊:
  • 影响因子:
    2.800
  • 作者:
    Huang Ham;Samuel D. McDougle;Anne G.E. Collins
  • 通讯作者:
    Anne G.E. Collins
A goal-centric outlook on learning
以目标为中心的学习观
  • DOI:
    10.1016/j.tics.2023.08.011
  • 发表时间:
    2023-12-01
  • 期刊:
  • 影响因子:
    17.200
  • 作者:
    Gaia Molinaro;Anne G.E. Collins
  • 通讯作者:
    Anne G.E. Collins

Anne G.E. Collins的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Anne G.E. Collins', 18)}}的其他基金

Thalamocortical cognitive networks in the healthy human brain
健康人脑中的丘脑皮质认知网络
  • 批准号:
    10633809
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Developing artificial neural network tools for cognitive modeling
开发用于认知建模的人工神经网络工具
  • 批准号:
    10641215
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
  • 批准号:
    10113371
  • 财政年份:
    2019
  • 资助金额:
    $ 38.01万
  • 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
  • 批准号:
    10576384
  • 财政年份:
    2019
  • 资助金额:
    $ 38.01万
  • 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
  • 批准号:
    9894854
  • 财政年份:
    2019
  • 资助金额:
    $ 38.01万
  • 项目类别:

相似海外基金

Impact of tissue resident memory T cells on the neuro-immune pathophysiology of anterior eye disease
组织驻留记忆 T 细胞对前眼疾病神经免疫病理生理学的影响
  • 批准号:
    10556857
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Anterior Insula Projections for Alcohol Drinking/Anxiety Interactions in Female and Male Rats
雌性和雄性大鼠饮酒/焦虑相互作用的前岛叶预测
  • 批准号:
    10608759
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Fear and anxiety circuit mechanisms in anterior hypothalamic nucleus
下丘脑前核的恐惧和焦虑环路机制
  • 批准号:
    10789153
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Elucidating signaling networks in Anterior Segment development, repair and diseases
阐明眼前节发育、修复和疾病中的信号网络
  • 批准号:
    10718122
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
The Intimate Interplay Between Keratoconus, Sex Hormones, and the Anterior Pituitary
圆锥角膜、性激素和垂体前叶之间的密切相互作用
  • 批准号:
    10746247
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Impact of tissue resident memory T cells on the neuro-immunepathophysiology of anterior eye disease
组织驻留记忆 T 细胞对前眼疾病神经免疫病理生理学的影响
  • 批准号:
    10804810
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
Investigation of the effect of anterior eye shape on myopia progression due to prolonged near work.
研究因长时间近距离工作而导致的前眼形状对近视进展的影响。
  • 批准号:
    23K09063
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Generation and characterization of anterior pituitary stem cells from human pluripotent stem cells
人多能干细胞垂体前叶干细胞的产生和表征
  • 批准号:
    23K08005
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Anterior cruciate ligament injury: towards a gendered environmental approach
前十字韧带损伤:走向性别环境方法
  • 批准号:
    485090
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Operating Grants
EASI-TOC: Endovascular Acute Stroke Intervention-Tandem OCclusion: atrial of acute cervical internal carotid artery stenting during endovascularthrombectomy for anterior circulation stroke
EASI-TOC:血管内急性卒中干预-串联闭塞:前循环卒中血管内血栓切除术期间急性颈内动脉心房支架置入术
  • 批准号:
    490056
  • 财政年份:
    2023
  • 资助金额:
    $ 38.01万
  • 项目类别:
    Operating Grants
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了