The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
基本信息
- 批准号:10576384
- 负责人:
- 金额:$ 37.94万
- 依托单位:
- 依托单位国家:美国
- 项目类别:
- 财政年份:2019
- 资助国家:美国
- 起止时间:2019-04-01 至 2025-01-31
- 项目状态:未结题
- 来源:
- 关键词:AnteriorAnxietyAreaArtificial IntelligenceAttention deficit hyperactivity disorderAutomobile DrivingBasal GangliaBehaviorBehavioralBrainClinicalCoffeeComplexComputer ModelsCorpus striatum structureCuriositiesDataDecision MakingDiagnosisDiseaseDopamineEducational process of instructingEnvironmentExperimental ModelsFunctional disorderGoalsHumanImpairmentIndividualIntelligenceLeadLearningLearning SkillLifeLightMental DepressionMental disordersModelingMotorOutcomeOutcomes ResearchPerformancePersonsPlayPopulationPrefrontal CortexProcessProtocols documentationPsyche structurePsychiatryPsychological reinforcementRainResearch Domain CriteriaRewardsRoleSchizophreniaSensorySignal TransductionSourceStructureSystemTestingThinkingVariantWaterWorkautism spectrum disorderbeancognitive systemexecutive functionexperimental studyflexibilityfunctional magnetic resonance imaging/electroencephalographyinsightlearning algorithmmathematical modelneuralneuromechanismnovelpreventskillstheoriestool
项目摘要
The neural computations supporting hierarchical reinforcement learning - Project Summary.
This project explores how humans learn at multiple hierarchical levels in parallel, and how this supports human
intelligence. Human decisions are typically hierarchically structured: we make high-level decisions (making a
cup of coffee), which constrain lower level decisions (grinding coffee beans, boiling water, etc.), which
themselves constrain simpler and simpler decisions and motor actions. This hierarchy in decisions is paralleled
by a hierarchy in our representation of our environment: some sensory signals trigger simple decisions (a red
light signals a stop), while other signal a broader, more abstract behavioral change (rain signals a set of
adaptations when driving). Thus, complex hierarchical structure underlies the way we respond to our
environment in seemingly simple, everyday tasks. This ability is supported by the prefrontal cortex, which
represents states and decisions at multiple degrees of hierarchical abstraction. My previous work shows that
hierarchical representations support transfer and generalization while learning, an ability that artificial agents
still struggle to match human performance in. However, how we learn to form these hierarchical
representations is poorly understood, despite how crucial it is for human intelligence. The proposed work will
examine how multiple, parallel hierarchical loops between prefrontal cortex and the basal ganglia support
reinforcement learning at multiple hierarchical levels in parallel, and how this promotes flexible behavior. To
this end, we will address three aims: 1. We will show that the same reinforcement learning computations
happen in parallel at multiple levels of abstraction, as hypothesized by our computational model of prefrontal-
subcortical networks. 2. We will demonstrate that humans partition learning problems into multiple sequential
subgoals so they can learn multiple simple strategies instead of one complex strategy, and that reusing these
simple strategies promotes fast exploration and learning. 3. We will show that hierarchical learning does not
rely exclusively on rewards, but that novelty signals are crucial for identifying subgoals and learning through
curiosity. Across all three aims, we will use behavioral experiments in conjunction with computational modeling
to characterize how humans learn hierarchically. In addition, we will use EEG and fMRI to identify the neural
computations underlying the cognitive systems inferred from behavior and modeling. This project will provide
new insights into the computational mechanisms that give rise to learning, and thus provide a better handle on
the sources of learning dysfunction observed in many psychiatric diseases, including schizophrenia,
depression, anxiety, ADHD, and OCD. Additionally, it will provide new tools, in the form of experimental
protocols and precise computational models, for studying learning across populations and species.
支持分层强化学习的神经计算-项目摘要。
项目成果
期刊论文数量(8)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Credit assignment in hierarchical option transfer
- DOI:
- 发表时间:2022-07
- 期刊:
- 影响因子:0
- 作者:Jing-Jing Li-Jing;Liyu Xia;Flora Dong;Anne G. E. Collins
- 通讯作者:Jing-Jing Li-Jing;Liyu Xia;Flora Dong;Anne G. E. Collins
Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making.
动态噪声估计:决策中噪声波动建模的通用方法。
- DOI:10.1101/2023.06.19.545524
- 发表时间:2024
- 期刊:
- 影响因子:0
- 作者:Li,Jing-Jing;Shi,Chengchun;Li,Lexin;Collins,AnneGE
- 通讯作者:Collins,AnneGE
How the Mind Creates Structure: Hierarchical Learning of Action Sequences.
思维如何创建结构:动作序列的分层学习。
- DOI:
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Eckstein,MariaK;Collins,AnneGE
- 通讯作者:Collins,AnneGE
Beyond dichotomies in reinforcement learning.
- DOI:10.1038/s41583-020-0355-6
- 发表时间:2020-10
- 期刊:
- 影响因子:0
- 作者:Collins AGE;Cockburn J
- 通讯作者:Cockburn J
Temporal and state abstractions for efficient learning, transfer, and composition in humans.
- DOI:10.1037/rev0000295
- 发表时间:2021-07
- 期刊:
- 影响因子:5.4
- 作者:Xia L;Collins AGE
- 通讯作者:Collins AGE
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Anne G.E. Collins其他文献
Dual effects of dual-tasking on instrumental learning
- DOI:
10.1016/j.cognition.2025.106228 - 发表时间:
2025-11-01 - 期刊:
- 影响因子:2.800
- 作者:
Huang Ham;Samuel D. McDougle;Anne G.E. Collins - 通讯作者:
Anne G.E. Collins
A goal-centric outlook on learning
以目标为中心的学习观
- DOI:
10.1016/j.tics.2023.08.011 - 发表时间:
2023-12-01 - 期刊:
- 影响因子:17.200
- 作者:
Gaia Molinaro;Anne G.E. Collins - 通讯作者:
Anne G.E. Collins
Anne G.E. Collins的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Anne G.E. Collins', 18)}}的其他基金
Thalamocortical cognitive networks in the healthy human brain
健康人脑中的丘脑皮质认知网络
- 批准号:
10633809 - 财政年份:2023
- 资助金额:
$ 37.94万 - 项目类别:
Developing artificial neural network tools for cognitive modeling
开发用于认知建模的人工神经网络工具
- 批准号:
10641215 - 财政年份:2023
- 资助金额:
$ 37.94万 - 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
- 批准号:
10359201 - 财政年份:2019
- 资助金额:
$ 37.94万 - 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
- 批准号:
10113371 - 财政年份:2019
- 资助金额:
$ 37.94万 - 项目类别:
The neural computations supporting hierarchical reinforcement learning
支持分层强化学习的神经计算
- 批准号:
9894854 - 财政年份:2019
- 资助金额:
$ 37.94万 - 项目类别:
相似海外基金
Atomic Anxiety in the New Nuclear Age: How Can Arms Control and Disarmament Reduce the Risk of Nuclear War?
新核时代的原子焦虑:军控与裁军如何降低核战争风险?
- 批准号:
MR/X034690/1 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Fellowship
Clinitouch-360: A digital health platform enabling robust end-to-end care of patients in Primary Care with depression and anxiety
Clinitouch-360:数字健康平台,可为初级保健中的抑郁和焦虑患者提供强大的端到端护理
- 批准号:
10098274 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Collaborative R&D
Mental Health and Occupational Functioning in Nurses: An investigation of anxiety sensitivity and factors affecting future use of an mHealth intervention
护士的心理健康和职业功能:焦虑敏感性和影响未来使用移动健康干预措施的因素的调查
- 批准号:
10826673 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Healthy Young Minds: co-producing a nature-based intervention with rural High School students to promote mental well-being and reduce anxiety
健康的年轻心灵:与农村高中生共同开展基于自然的干预措施,以促进心理健康并减少焦虑
- 批准号:
MR/Z503599/1 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Research Grant
Visual analysis system to detect and predict the signs of anxiety in healthcare
用于检测和预测医疗保健中焦虑迹象的视觉分析系统
- 批准号:
2902083 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Studentship
Using generative AI combined with immersive technology to treat anxiety disorders
利用生成式人工智能结合沉浸式技术治疗焦虑症
- 批准号:
10109165 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Launchpad
"Flashforward" imagery and anxiety in young adults: Risk mechanisms and intervention development
年轻人的“闪现”意象和焦虑:风险机制和干预措施的发展
- 批准号:
MR/Y009460/1 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Fellowship
How parents manage climate anxiety: coping and hoping for the whole family
父母如何应对气候焦虑:全家人的应对和希望
- 批准号:
DP230101928 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Discovery Projects
An innovative biofeedback enhanced adaptive extended reality (XR) device to reduce perinatal pain and anxiety during and after childbirth
一种创新的生物反馈增强型自适应扩展现实 (XR) 设备,可减少分娩期间和分娩后的围产期疼痛和焦虑
- 批准号:
10097862 - 财政年份:2024
- 资助金额:
$ 37.94万 - 项目类别:
Collaborative R&D
Application name Phase Space - VR hypnotherapy as early intervention for anxiety in students and young people
应用程序名称 Phase Space - VR 催眠疗法作为学生和年轻人焦虑的早期干预
- 批准号:
10055011 - 财政年份:2023
- 资助金额:
$ 37.94万 - 项目类别:
Collaborative R&D














{{item.name}}会员




