CRCNS: Reward and motivation in neural networks
CRCNS:神经网络中的奖励和动机
基本信息
- 批准号:9916069
- 负责人:
- 金额:$ 43.2万
- 依托单位:
- 依托单位国家:美国
- 项目类别:
- 财政年份:2019
- 资助国家:美国
- 起止时间:2019-09-30 至 2024-07-31
- 项目状态:已结题
- 来源:
- 关键词:Adaptive BehaviorsAnimalsBehaviorBehavioralClinicalComplexComputer AnalysisComputer SimulationComputing MethodologiesDataDrug AddictionElectrophysiology (science)Functional disorderGlobus PallidusGlutamatesGoalsHumanHungerImageImpairmentIndividualInstructionKnowledgeLearningLesionLinkMaintenanceMathematicsMental DepressionMethodsMolecular GeneticsMotivationMusNeural Network SimulationNeuronsNutrientOutcomePharmacologyPhysiologicalPlayPopulationPrincipal InvestigatorPsychological reinforcementPunishmentRecurrenceResearchRewardsRoleSignal TransductionStructureTestingThirstTrainingaddictionbasebehavior testcomputational neuroscienceexpectationflexibilityimprovedin vivoincentive saliencelearning networkmental statemotivated behaviormotivational processesneural networkneuromechanismnoveloptogeneticsprogramsresponsetheoriestool
项目摘要
The overall goal of this project is to develop a reinforcement learning (RL) theory of motivation, understood
here as motivational salience, and to test the conclusions of this theory using experimental observations
obtained in the ventral pallidum (VP). Animals' actions depend on the shifting values of internal demands
determined by physiological or behavioral conditions, such as thirst, hunger, addiction, specific nutrient
deficiency, etc. These need-based modulations of the perceived values of reinforcements (reward or
punishment} are described by a mathematical variable called motivational salience or, simply, motivation.
Including motivation adds a new level of complexity to RL theory, and allows it to generate flexible ongoing
behaviors. Here, we will investigate how motivation can be learned by neuronal networks to generate
complex adaptive behaviors and compare the conclusions of our theory with the VP circuits. Previous studies
indicate that the VP plays an important role in a variety of behaviors, potentially, by influencing motivational
salience. In vivo recordings suggest that VP neuron firing correlates with motivational states. Lesions,
pharmacological and optogenetic manipulations in VP cause profound changes in behaviors motivated by
natural rewards or drugs of addiction. Dysfunction of this structure is linked to depression and drug addiction
in humans. Our theoretical results suggest that distinct classes of neurons in the VP should play essential
roles in representing either positive or negative motivational states. We further hypothesize that the functional
interactions locally within the VP are critical for generating such signals that guide motivated behaviors.
Consistent with predictions of RL theory, in our preliminary studies, we found that individual VP neurons
could be classified as either positive or negative 'motivation neurons', as the activities of these neurons
represented both expected values of outcomes and motivational states. When population activity is
considered, representations of outcome expectation can be distinguished from representations of motivation
fluctuating according to the animals' physiological states. Based on the preliminary data, we devised an
integrated approach, combining studies in computational analysis and theory (Koulakov lab) with advanced
molecular genetic tools, optogenetics, chemogenetics, electrophysiology, and imaging in behaving mice (Li
lab), to test our hypotheses through the following Aims: Aim 1. To develop methods for identifying motivation
in the population activity of VP neurons. Here we will use novel behavioral and computational methods to
disambiguate representations of motivation and outcome expectation in neuronal responses. Aim 2. To
develop reinforcement learning theory of motivation and to test its predictions using responses of VP neurons.
Here we will develop the Q-learning theory of motivation and compare networks trained using this theory to
responses of VP neurons. Aim 3. To identify the circuit basis of representations of motivation in VP neuronal
populations. We will identify the network structure in Q-learning networks with motivation, and test predictions
using opto- and chemogenetic manipulations in VP.
RELEVANCE (See instructions):
The neural mechanisms of motivated behaviors remain unclear. In the proposed research program, we will
determine the precise circuit mechanisms and computations by which neurons in the ventral pallidum
participate in modulating motivated behaviors. Findings from this project will have important clinical
implications, as impairments in motivational processes are core features of depression and drug addiction.
本项目的总体目标是开发激励的强化学习(RL)理论,了解
这里作为动机的突显,并用实验观察来检验这一理论的结论
取自腹侧苍白球(VP)。动物的行为取决于内在需求价值的变化
由生理或行为条件决定的,如口渴、饥饿、上瘾、特定的营养
不足等。这些基于需要的增援感知价值的调制(奖励或
惩罚是由一个数学变量来描述的,这个变量被称为动机显著,或者简单地说,动机。
包括动机将RL理论的复杂性提高到一个新的水平,并使其能够产生灵活的持续
行为。在这里,我们将研究如何通过神经网络学习动机来产生
复杂的自适应行为,并与VP电路的结论进行了比较。以前的研究
指出VP通过潜在地影响动机,在各种行为中起着重要作用
显著程度。活体记录表明,VP神经元的放电与动机状态有关。损伤,
VP的药理和光遗传操作引起行为的深刻变化
自然的奖励或上瘾的药物。这一结构的功能障碍与抑郁症和药物成瘾有关
在人类身上。我们的理论结果表明,VP中不同类别的神经元应该发挥重要作用
在代表积极或消极激励状态方面的作用。我们进一步假设泛函
VP内部的局部相互作用对于产生引导动机行为的信号至关重要。
与RL理论的预测一致,在我们的初步研究中,我们发现单个VP神经元
可以被归类为积极或消极的‘动机神经元’,因为这些神经元的活动
代表了结果的期望值和激励状态。当人口活动达到
考虑到,结果期望的表征可以与动机的表征区分开来
根据动物的生理状态波动。根据初步数据,我们设计了一个
综合方法,将计算分析和理论研究(Koulakov实验室)与高级
分子遗传学工具、光遗传学、化学遗传学、电生理学和行为小鼠的成像
实验),通过以下目标验证我们的假设:目标1.开发识别动机的方法
在VP神经元的群体活动中。在这里,我们将使用新的行为和计算方法
消除神经元反应中动机和结果预期的歧义表达。目标2.目标
发展动机的强化学习理论,并利用VP神经元的反应来检验其预测。
在这里,我们将发展动机的Q学习理论,并将使用该理论训练的网络与
VP神经元的反应。目的3.确定VP神经元中动机表征的回路基础
人口。我们将用动机识别Q-学习网络中的网络结构,并测试预测
在VP中使用光和化学发生操作。
相关性(请参阅说明):
动机行为的神经机制仍不清楚。在拟议的研究计划中,我们将
确定腹侧苍白球神经元的精确回路机制和计算
参与调整受激励的行为。该项目的发现将具有重要的临床意义
影响,因为动机过程中的障碍是抑郁和吸毒成瘾的核心特征。
项目成果
期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
数据更新时间:{{ journalArticles.updateTime }}
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
ALEXEI KOULAKOV其他文献
ALEXEI KOULAKOV的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('ALEXEI KOULAKOV', 18)}}的其他基金
CRCNS: Reward and motivation in neural networks
CRCNS:神经网络中的奖励和动机
- 批准号:
10455096 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
CRCNS: Reward and motivation in neural networks
CRCNS:神经网络中的奖励和动机
- 批准号:
10017031 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
Predictive Computational Models of Olfactory Networks
嗅觉网络的预测计算模型
- 批准号:
10200170 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
CRCNS: Reward and motivation in neural networks
CRCNS:神经网络中的奖励和动机
- 批准号:
10227072 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
CRCNS: Reward and motivation in neural networks
CRCNS:神经网络中的奖励和动机
- 批准号:
10675602 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
Predictive Computational Models of Olfactory Networks
嗅觉网络的预测计算模型
- 批准号:
10670089 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
Predictive Computational Models of Olfactory Networks
嗅觉网络的预测计算模型
- 批准号:
10413210 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
CRCNS: Sparse odor coding in the olfactory bulb
CRCNS:嗅球中的稀疏气味编码
- 批准号:
9066624 - 财政年份:2014
- 资助金额:
$ 43.2万 - 项目类别:
CRCNS: Sparse odor coding in the olfactory bulb
CRCNS:嗅球中的稀疏气味编码
- 批准号:
8837253 - 财政年份:2014
- 资助金额:
$ 43.2万 - 项目类别:
CRCNS: Theory and experiment of neural circuit mapping by DNA sequencing
CRCNS:DNA测序神经回路图谱的理论与实验
- 批准号:
9246516 - 财政年份:2013
- 资助金额:
$ 43.2万 - 项目类别:
相似海外基金
CAREER: Next-generation of Wirelessly Powered Implantable Neuromodulation and Electrophysiological Recording System for Long-term Behavior Study of Freely-Moving Animals
职业:下一代无线供电植入式神经调节和电生理记录系统,用于自由移动动物的长期行为研究
- 批准号:
2309413 - 财政年份:2022
- 资助金额:
$ 43.2万 - 项目类别:
Continuing Grant
Developing remote monitoring system of aquatic animals' behavior and ecology to reform ecosystem conservation
开发水生动物行为和生态远程监测系统改革生态系统保护
- 批准号:
22K18432 - 财政年份:2022
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Challenging Research (Pioneering)
OCE-PRF: Cliff Hangers: Investigating Effects of a Submarine Canyon on the Distribution and Behavior of Midwater Animals and their Predators
OCE-PRF:悬崖吊架:调查海底峡谷对中层水域动物及其捕食者的分布和行为的影响
- 批准号:
2126537 - 财政年份:2021
- 资助金额:
$ 43.2万 - 项目类别:
Standard Grant
CAREER: Next-generation of Wirelessly Powered Implantable Neuromodulation and Electrophysiological Recording System for Long-term Behavior Study of Freely-Moving Animals
职业:下一代无线供电植入式神经调节和电生理记录系统,用于自由移动动物的长期行为研究
- 批准号:
1943990 - 财政年份:2020
- 资助金额:
$ 43.2万 - 项目类别:
Continuing Grant
Study on factors that increase or decrease the vigilance behavior of wild animals: the effect of species differences and visual stimuli
野生动物警觉行为增减因素研究:物种差异和视觉刺激的影响
- 批准号:
20K06353 - 财政年份:2020
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Scientific Research (C)
Neural circuit underlying flexible behavior in animals
动物灵活行为的神经回路
- 批准号:
19H01769 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Scientific Research (B)
Analysis of adaptive mechanisms in chemical localization behavior of animals by using novel devices to intervene in sensory and motor functions
使用新型装置干预感觉和运动功能来分析动物化学定位行为的适应性机制
- 批准号:
19H02104 - 财政年份:2019
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Scientific Research (B)
Life Cost Strategy for Wild Animals Using Wearable Behavior Recording Devices and Telomere Measurement
使用可穿戴行为记录设备和端粒测量的野生动物生命成本策略
- 批准号:
18K14788 - 财政年份:2018
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Early-Career Scientists
Modeling and application of energy-efficient behavior in calling animals
动物呼叫节能行为建模及应用
- 批准号:
18K18005 - 财政年份:2018
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Early-Career Scientists
Cooperative behavior of non-human animals focusing on reward sharing -comparison between rodents and birds-
注重奖励分享的非人类动物的合作行为-啮齿类动物与鸟类的比较-
- 批准号:
18K12020 - 财政年份:2018
- 资助金额:
$ 43.2万 - 项目类别:
Grant-in-Aid for Scientific Research (C)