权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Instrumental divergence and goal-directed choice

职业：工具分歧和目标导向的选择

基本信息

批准号：
1654187
负责人：
Mimi Liljeholm
金额：
$ 72.77万
依托单位：
University of California-Irvine
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-02-15 至 2025-01-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1654187&HistoricalAwards=false
关键词：
CAREER Instrumental divergence goal directed

项目摘要

Theories of instrumental behavior distinguish between goal-directed decisions, motivated by a deliberate consideration of the probability and current utility of their consequences, and habits, which are rigidly and automatically elicited by the stimulus environment based on reinforcement history. In spite of the far-reaching implications of this distinction, ranging from the structuring of economic policies to the diagnosis and treatment of behavioral pathology, much is still unknown about what factors shape goal-directed decisions and what conditions prompt a transition from goal-directed to habitual action selection. Generally, while computationally expensive, a goal-directed strategy offers greater levels of flexible instrumental control. Since subjective utilities often change from one moment to the next, such flexibility is essential for reward maximization and thus may have intrinsic value, potentially serving to motivate and reinforce specific decisions, as well as to justify the general processing cost of goal-directed computations. A critical requirement for flexible instrumental control, however, is that available action alternatives yield distinct outcome states. With the support of this NSF Career award, Dr. Mimi Liljeholm is investigating the novel hypotheses that instrumental divergence? the difference between outcome probability distributions associated with alternative actions? can shape choice preferences, induce conditioned reinforcement, and arbitrate between goal-directed and habitual decision strategies. The objective of this research is to address important gaps in current knowledge about the nature and limits of goal-directed behavior, using a combination of innovative experimental designs, computational modeling and functional magnetic resonance imaging (fMRI). The educational component of the award provides hands-on training in neuroimaging methods, and in the computational and neural bases of learning and decision-making, at undergraduate and graduate levels. All studies use a simple gambling task in which alternative actions yield different colored tokens, each worth a particular amount of money, with various probabilities. In studies assessing a preference for flexible instrumental control, the relevant choice is between pairs of actions with different levels of instrumental divergence. Expected monetary pay-offs vary independently of instrumental divergence across options, dissociating the relative contribution of each factor to behavioral choice performance. Studies investigating the capacity of high instrumental divergence to induce conditioned reinforcement measure changes in the affective valence of visual stimuli based on their association with high versus low instrumental divergence. Finally, following extended exposure to high versus low instrumental divergence, the degree to which behavior is goal-directed or habitual is assessed using a standardly employed outcome devaluation procedure, in which the monetary amount associated with a particular token color is altered: Goal-directed, but not habitual, decisions are modulated by such changes in the utility of sensory-specific outcomes states. Neuroimaging data is acquired by scanning participants with fMRI as they perform the task, and a reinforcement learning framework is used to model the intrinsic value of flexible instrumental control (by treating instrumental divergence as a surrogate reward) at behavioral and neural levels. Since many psychiatric disorders are characterized by an abnormal sense of agency, and addiction associated with a rapid transition from goal-directed to habitual action-selection, broader impacts of this project include the potential development of pre-clinical diagnostic assays for early detection of cognitive, affective and behavioral pathology. The concepts advanced under this project may also help improve the performance of reinforcement learning algorithms, for example by using instrumental divergence to specify new optimization criteria, potentially benefiting medical, industrial and commercial applications of artificial intelligence.

工具性行为理论区分了目标导向决策和习惯，前者是出于对其后果的概率和当前效用的刻意考虑，后者是由基于强化历史的刺激环境僵化和自动引发的。尽管这种区别具有深远的影响，从经济政策的结构到行为病理学的诊断和治疗，但对于哪些因素影响目标导向型决策，以及哪些条件促使从目标导向型选择向习惯性行动选择转变，仍有许多未知之处。一般来说，虽然计算成本很高，但目标导向策略提供了更高水平的灵活工具控制。由于主观效用经常在不同时刻发生变化，这种灵活性对于报酬最大化是必不可少的，因此可能具有内在价值，潜在地用于激励和强化特定决策，以及证明目标导向计算的总体处理成本是合理的。然而，灵活的仪器控制的一个关键要求是，可用的行动替代方案产生不同的结果状态。在NSF职业生涯奖的支持下，米米·利尔杰霍姆博士正在研究工具性分歧？与替代行动相关的结果概率分布之间的差异？可以塑造选择偏好，诱导条件性强化，并在目标导向和习惯性决策策略之间进行仲裁。这项研究的目的是利用创新的实验设计、计算模型和功能磁共振成像(FMRI)相结合的方法，解决目前关于目标导向行为的性质和限制的重要知识空白。该奖项的教育部分提供本科生和研究生级别的神经成像方法以及学习和决策的计算和神经基础方面的实践培训。所有研究都使用一个简单的赌博任务，在这个任务中，不同的行动产生不同颜色的代币，每个代币价值特定的金额，具有不同的概率。在评估灵活工具控制偏好的研究中，相关的选择是在工具差异程度不同的一对动作之间进行的。预期的货币回报与不同期权之间的工具差异无关，分离了每个因素对行为选择绩效的相对贡献。研究了高工具发散诱导条件性强化的能力，根据高工具发散与低工具发散的关联来测量视觉刺激的情感效价的变化。最后，在长期暴露于高和低工具差异之后，使用标准使用的结果贬值程序来评估行为是目标导向的或习惯性的程度，其中与特定标记颜色相关的货币金额被改变：目标导向但不是习惯性的决定受到特定感觉结果状态效用的这种变化的调节。神经成像数据是通过在参与者执行任务时用fMRI扫描他们来获得的，强化学习框架被用来在行为和神经水平上模拟灵活工具控制的内在价值(通过将工具差异视为替代奖励)。由于许多精神障碍的特点是不正常的代理感，以及与从目标导向到习惯性行动选择的快速转变相关的成瘾，该项目的更广泛影响包括潜在的临床前诊断分析的发展，以早期发现认知、情感和行为病理。在该项目下提出的概念也可能有助于提高强化学习算法的性能，例如通过使用工具发散来指定新的优化标准，潜在地有利于人工智能的医疗、工业和商业应用。