Affective arousal as a guide for reinforcement learning in multidimensional environments

情感唤醒作为多维环境中强化学习的指南

• 批准号: 10641795
• 负责人: David E Melnikoff
• 金额: $ 2.15万
• 依托单位: NORTHEASTERN UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-05-31 至 2023-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10641795
• 关键词: Address; Adult; Affect; Affective; Arousal; Attention; Basic Science; Beauty; Behavior; Behavioral; Biological; Brain; Complex; Data; Dimensions; Economic Burden; Environment; Fellowship; Investments; Knowledge; Learning; Link; Literature; Location; Measurement; Mediating; Memory; Memory impairment; Modeling; Mood Disorders; National Research Service Awards; Neurosciences; Participant; Persons; Play; Prevention strategy; Process; Psychological reinforcement; Psychology; Research; Resources; Role; Signal Transduction; Stimulus; Testing; Time; Translating; Weight; blood oxygen level dependent; career; clinical application; computational neuroscience; computer framework; cost; directed attention; experimental study; improved; innovation; learning strategy; neural; neuroimaging; psychologic; skills; treatment strategy; two-dimensional

Project Summary/Abstract In 2020, affective disorders are expected to impact over 35 million U.S. adults, imposing an economic burden in excess of $200 billion, and exacting an immeasurable personal toll on those affected. To mitigate these costs, the fields of neuroscience and psychology have devoted considerable resources to basic research aimed at identifying vulnerabilities, treatments, and prevention strategies for affective disorders. These investments have been slow to translate into concrete improvements, however, due in part to the enormous explanatory gap between neural and mentalistic accounts of affect. Faster progress requires a bridge from measurements at the biological level to phenomena at the psychological level. With a three-year NRSA fellowship, I will begin to develop this bridge by studying affect within the framework of computational reinforcement learning (RL), which has emerged as an indispensable guide for linking neural activity to psychological phenomena. I propose to use innovative RL models to bridge behavioral and biological data from two neuroimaging studies, both of which focus on affect and its role in attention and learning. These studies address a deep psychological problem: At any moment, the options for what a brain can attend to and learn about are infinite, so how do brains decide what to focus on and what to ignore? Consider, for example a seemingly simple task commonly encountered in neuroscience experiments: learning associations between words and pictures of scenes. Scenes vary along countless dimensions (location, habitability, beauty, etc.), so how do brains decide which dimensions to associate with the words, and which to ignore? I propose that such decisions are guided by affective arousal. Specifically, I hypothesize that high arousal directs attention towards dimensions that can take on a small number of values (e.g., whether the scene is indoors or outdoors, a dimension that can take on just two values). I call these low-cardinality dimensions, in contrast to high- cardinality dimensions, which can take on many values (e.g., the specific location of the scene). My hypothesis builds on research showing that brains in states of high arousal opt for fast, efficient learning strategies; all else being equal, low- cardinality dimensions are relatively easy to learn about, and high-cardinality dimensions are relatively difficult to learn about, so high arousal should direct attention towards the former and away from the latter. Testing this hypothesis will help launch my career at the intersection of psychology and computational neuroscience — a transdisciplinary approach that, I believe, will play an essential role in translating basic research into clinical applications. Specifically, I will develop the computational and neuroscience skills necessary to bridge the explanatory gap between neural and mentalistic accounts of affect, thereby aiding in the understanding, prediction, and treatment of affective disorders.

项目总结/摘要 到2020年，情感障碍预计将影响超过3500万美国成年人，造成的经济负担将超过 2000亿美元，并对受影响的人造成不可估量的个人损失。为了降低这些成本， 神经科学和心理学投入了大量资源用于旨在识别脆弱性的基础研究， 情感障碍的治疗和预防策略。这些投资转化为具体成果的速度缓慢 然而，这种改善部分是由于神经和心理对情感的解释之间存在巨大的差距。 更快的进展需要从生物水平的测量到心理水平的现象的桥梁。 有了三年的NRSA奖学金，我将开始通过在以下框架内研究影响来发展这座桥梁： 计算强化学习（RL），它已成为将神经活动与 心理现象。我建议使用创新的强化学习模型来桥接来自两个方面的行为和生物数据， 神经影像学研究，两者都侧重于情感及其在注意力和学习中的作用。这些研究探讨了一个深刻的 心理问题：在任何时候，大脑可以关注和学习的选择都是无限的，那么如何 大脑决定关注什么忽略什么例如，考虑一个通常遇到的看似简单的任务 在神经科学实验中：学习单词和场景图片之间的联系。场景变化沿着无数 维度（位置、可居住性、美观性等），那么大脑是如何决定哪些维度与单词相关联的呢， 哪些可以忽略？我认为，这些决定是由情感唤醒引导的。具体来说，我假设 唤醒将注意力引导到可以采用少量值的维度（例如，无论场景是在室内 或室外，可以仅取两个值的维度）。我称之为低基数维度，与高基数维度相反， 基数维度，其可以具有许多值（例如，场景的具体位置）。我的假设建立在 研究表明，大脑在高唤醒状态下选择快速，有效的学习策略;其他条件相同，低- 基数维相对容易学习，而高基数维相对较难学习 因此，高唤醒应该将注意力引向前者，远离后者。检验这一假设将 帮助我在心理学和计算神经科学的交叉点上开始我的职业生涯-跨学科方法 我相信，这将在基础研究转化为临床应用方面发挥重要作用。具体来说，我将开发 计算和神经科学的技能，必要的桥梁之间的解释差距神经和心灵 情感的帐户，从而帮助理解，预测和治疗情感障碍。