权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

The neural computations supporting hierarchical reinforcement learning

支持分层强化学习的神经计算

基本信息

批准号：
10576384
负责人：
Anne G.E. Collins
金额：
$ 37.94万
依托单位：
UNIVERSITY OF CALIFORNIA BERKELEY
依托单位国家：
美国
项目类别：
财政年份：
2019
资助国家：
美国
起止时间：
2019-04-01 至 2025-01-31
项目状态：
未结题

来源：
https://reporter.nih.gov/project-details/10576384
关键词：
Anterior Anxiety Area Artificial Intelligence Attention deficit hyperactivity disorder Automobile Driving Basal Ganglia Behavior Behavioral Brain Clinical Coffee Complex Computer Models Corpus striatum structure Curiosities Data Decision Making Diagnosis Disease Dopamine Educational process of instructing Environment Experimental Models Functional disorder Goals Human Impairment Individual Intelligence Lead Learning Learning Skill Life Light Mental Depression Mental disorders Modeling Motor Outcome Outcomes Research Performance Persons Play Population Prefrontal Cortex Process Protocols documentation Psyche structure Psychiatry Psychological reinforcement Rain Research Domain Criteria Rewards Role Schizophrenia Sensory Signal Transduction Source Structure System Testing Thinking Variant Water Work autism spectrum disorder bean cognitive system executive function experimental study flexibility functional magnetic resonance imaging/electroencephalography insight learning algorithm mathematical model neural neuromechanism novel prevent skills theories tool

项目摘要

The neural computations supporting hierarchical reinforcement learning - Project Summary. This project explores how humans learn at multiple hierarchical levels in parallel, and how this supports human intelligence. Human decisions are typically hierarchically structured: we make high-level decisions (making a cup of coffee), which constrain lower level decisions (grinding coffee beans, boiling water, etc.), which themselves constrain simpler and simpler decisions and motor actions. This hierarchy in decisions is paralleled by a hierarchy in our representation of our environment: some sensory signals trigger simple decisions (a red light signals a stop), while other signal a broader, more abstract behavioral change (rain signals a set of adaptations when driving). Thus, complex hierarchical structure underlies the way we respond to our environment in seemingly simple, everyday tasks. This ability is supported by the prefrontal cortex, which represents states and decisions at multiple degrees of hierarchical abstraction. My previous work shows that hierarchical representations support transfer and generalization while learning, an ability that artificial agents still struggle to match human performance in. However, how we learn to form these hierarchical representations is poorly understood, despite how crucial it is for human intelligence. The proposed work will examine how multiple, parallel hierarchical loops between prefrontal cortex and the basal ganglia support reinforcement learning at multiple hierarchical levels in parallel, and how this promotes flexible behavior. To this end, we will address three aims: 1. We will show that the same reinforcement learning computations happen in parallel at multiple levels of abstraction, as hypothesized by our computational model of prefrontal- subcortical networks. 2. We will demonstrate that humans partition learning problems into multiple sequential subgoals so they can learn multiple simple strategies instead of one complex strategy, and that reusing these simple strategies promotes fast exploration and learning. 3. We will show that hierarchical learning does not rely exclusively on rewards, but that novelty signals are crucial for identifying subgoals and learning through curiosity. Across all three aims, we will use behavioral experiments in conjunction with computational modeling to characterize how humans learn hierarchically. In addition, we will use EEG and fMRI to identify the neural computations underlying the cognitive systems inferred from behavior and modeling. This project will provide new insights into the computational mechanisms that give rise to learning, and thus provide a better handle on the sources of learning dysfunction observed in many psychiatric diseases, including schizophrenia, depression, anxiety, ADHD, and OCD. Additionally, it will provide new tools, in the form of experimental protocols and precise computational models, for studying learning across populations and species.

支持分层强化学习的神经计算-项目摘要。

项目成果

期刊论文数量（8）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Credit assignment in hierarchical option transfer

DOI：
发表时间：
2022-07
期刊：
CogSci ... Annual Conference of the Cognitive Science Society. Cognitive Science Society (U.S.). Conference
影响因子：
0
作者：
Jing-Jing Li-Jing;Liyu Xia;Flora Dong;Anne G. E. Collins
通讯作者：
Jing-Jing Li-Jing;Liyu Xia;Flora Dong;Anne G. E. Collins

Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making.

动态噪声估计：决策中噪声波动建模的通用方法。