The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
基本信息
- 批准号:10359201
- 负责人:
- 金额:$ 38.01万
- 依托单位:
- 依托单位国家:美国
- 项目类别:
- 财政年份:2019
- 资助国家:美国
- 起止时间:2019-04-01 至 2024-01-31
- 项目状态:已结题
- 来源:
- 关键词:AddressAnteriorAnxietyAreaArtificial IntelligenceAttention deficit hyperactivity disorderAutomobile DrivingBasal GangliaBehaviorBehavioralBrainClinicalCoffeeComplexComputer ModelsCorpus striatum structureCuriositiesDataDecision MakingDiagnosisDiseaseDopamineEducational process of instructingElectroencephalographyEnvironmentExperimental ModelsFunctional Magnetic Resonance ImagingFunctional disorderGoalsHumanImpairmentIndividualIntelligenceLeadLearningLifeLightMental DepressionMental disordersModelingMotorOutcomeOutcomes ResearchPerformancePersonsPlayPopulationPrefrontal CortexProcessProtocols documentationPsyche structurePsychiatryPsychological reinforcementRainResearch Domain CriteriaRewardsRoleSchizophreniaSensorySignal TransductionSourceStructureSystemTestingThinkingVariantWaterWorkautism spectrum disorderbasebeancognitive systemexecutive functionexperimental studyflexibilityinsightlearning algorithmmathematical modelneuromechanismnovelpreventrelating to nervous systemskillstheoriestool
项目摘要
The neural computations supporting hierarchical reinforcement learning - Project Summary.
This project explores how humans learn at multiple hierarchical levels in parallel, and how this supports human
intelligence. Human decisions are typically hierarchically structured: we make high-level decisions (making a
cup of coffee), which constrain lower level decisions (grinding coffee beans, boiling water, etc.), which
themselves constrain simpler and simpler decisions and motor actions. This hierarchy in decisions is paralleled
by a hierarchy in our representation of our environment: some sensory signals trigger simple decisions (a red
light signals a stop), while other signal a broader, more abstract behavioral change (rain signals a set of
adaptations when driving). Thus, complex hierarchical structure underlies the way we respond to our
environment in seemingly simple, everyday tasks. This ability is supported by the prefrontal cortex, which
represents states and decisions at multiple degrees of hierarchical abstraction. My previous work shows that
hierarchical representations support transfer and generalization while learning, an ability that artificial agents
still struggle to match human performance in. However, how we learn to form these hierarchical
representations is poorly understood, despite how crucial it is for human intelligence. The proposed work will
examine how multiple, parallel hierarchical loops between prefrontal cortex and the basal ganglia support
reinforcement learning at multiple hierarchical levels in parallel, and how this promotes flexible behavior. To
this end, we will address three aims: 1. We will show that the same reinforcement learning computations
happen in parallel at multiple levels of abstraction, as hypothesized by our computational model of prefrontal-
subcortical networks. 2. We will demonstrate that humans partition learning problems into multiple sequential
subgoals so they can learn multiple simple strategies instead of one complex strategy, and that reusing these
simple strategies promotes fast exploration and learning. 3. We will show that hierarchical learning does not
rely exclusively on rewards, but that novelty signals are crucial for identifying subgoals and learning through
curiosity. Across all three aims, we will use behavioral experiments in conjunction with computational modeling
to characterize how humans learn hierarchically. In addition, we will use EEG and fMRI to identify the neural
computations underlying the cognitive systems inferred from behavior and modeling. This project will provide
new insights into the computational mechanisms that give rise to learning, and thus provide a better handle on
the sources of learning dysfunction observed in many psychiatric diseases, including schizophrenia,
depression, anxiety, ADHD, and OCD. Additionally, it will provide new tools, in the form of experimental
protocols and precise computational models, for studying learning across populations and species.
支持分层强化学习的神经计算-项目摘要。
项目成果
期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
数据更新时间:{{ journalArticles.updateTime }}
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Anne G.E. Collins其他文献
Dual effects of dual-tasking on instrumental learning
- DOI:
10.1016/j.cognition.2025.106228 - 发表时间:
2025-11-01 - 期刊:
- 影响因子:2.800
- 作者:
Huang Ham;Samuel D. McDougle;Anne G.E. Collins - 通讯作者:
Anne G.E. Collins
A goal-centric outlook on learning
以目标为中心的学习观
- DOI:
10.1016/j.tics.2023.08.011 - 发表时间:
2023-12-01 - 期刊:
- 影响因子:17.200
- 作者:
Gaia Molinaro;Anne G.E. Collins - 通讯作者:
Anne G.E. Collins
Anne G.E. Collins的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Anne G.E. Collins', 18)}}的其他基金
Thalamocortical cognitive networks in the healthy human brain
健康人脑中的丘脑皮质认知网络
- 批准号:
10633809 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Developing artificial neural network tools for cognitive modeling
开发用于认知建模的人工神经网络工具
- 批准号:
10641215 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
- 批准号:
10113371 - 财政年份:2019
- 资助金额:
$ 38.01万 - 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
- 批准号:
10576384 - 财政年份:2019
- 资助金额:
$ 38.01万 - 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
- 批准号:
9894854 - 财政年份:2019
- 资助金额:
$ 38.01万 - 项目类别:
相似海外基金
Impact of tissue resident memory T cells on the neuro-immune pathophysiology of anterior eye disease
组织驻留记忆 T 细胞对前眼疾病神经免疫病理生理学的影响
- 批准号:
10556857 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Anterior Insula Projections for Alcohol Drinking/Anxiety Interactions in Female and Male Rats
雌性和雄性大鼠饮酒/焦虑相互作用的前岛叶预测
- 批准号:
10608759 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Fear and anxiety circuit mechanisms in anterior hypothalamic nucleus
下丘脑前核的恐惧和焦虑环路机制
- 批准号:
10789153 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Elucidating signaling networks in Anterior Segment development, repair and diseases
阐明眼前节发育、修复和疾病中的信号网络
- 批准号:
10718122 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
The Intimate Interplay Between Keratoconus, Sex Hormones, and the Anterior Pituitary
圆锥角膜、性激素和垂体前叶之间的密切相互作用
- 批准号:
10746247 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Impact of tissue resident memory T cells on the neuro-immunepathophysiology of anterior eye disease
组织驻留记忆 T 细胞对前眼疾病神经免疫病理生理学的影响
- 批准号:
10804810 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Investigation of the effect of anterior eye shape on myopia progression due to prolonged near work.
研究因长时间近距离工作而导致的前眼形状对近视进展的影响。
- 批准号:
23K09063 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
Generation and characterization of anterior pituitary stem cells from human pluripotent stem cells
人多能干细胞垂体前叶干细胞的产生和表征
- 批准号:
23K08005 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
Anterior cruciate ligament injury: towards a gendered environmental approach
前十字韧带损伤:走向性别环境方法
- 批准号:
485090 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Operating Grants
EASI-TOC: Endovascular Acute Stroke Intervention-Tandem OCclusion: atrial of acute cervical internal carotid artery stenting during endovascularthrombectomy for anterior circulation stroke
EASI-TOC:血管内急性卒中干预-串联闭塞:前循环卒中血管内血栓切除术期间急性颈内动脉心房支架置入术
- 批准号:
490056 - 财政年份:2023
- 资助金额:
$ 38.01万 - 项目类别:
Operating Grants