Doctoral Dissertation Research In DRMS: Reinforcement Learning and Attention in Decision Making

决策者经常考虑来自各种来源的信息，但由于处理信息是费力的，并不是所有的信息都被考虑在内。这个项目调查了两个主要信息来源对决策的贡献：任何关于行动将产生的奖励的可用信息，以及过去采取该行动所获得的奖励历史。决策者在多大程度上依赖于这两种信息来源中的任何一种，对于理解通常观察到的节省学习的选择模式是至关重要的：当前的信息经常（至少部分地）被忽略，并被依赖先前的经验所取代。经济学的最新进展对这种权衡进行了建模，但通常不提供对先前经验的解释。然而，神经科学和人工智能提供了从先前的经验中学习的解释，例如，这与习惯的形成有关。该项目旨在将神经科学和经济理论结合在一个连贯的理论框架中，并通过实验量化决策分别依赖于当前信息和先前经验的程度。这样的测量对于理解决策者重新评估重复决策和打破习惯的情况是至关重要的。它将提出政策建议，帮助激励决策者做出深思熟虑的决定，而不是遵循持久的习惯。这一点非常重要，因为习惯在焦虑障碍、成瘾以及消费决策中发挥着核心作用。当前信息和先前经验的相对贡献在一个实验中进行评估，该实验牢固地植根于理性注意力不集中的经济理论以及认知神经科学强化学习的计算模型。该实验将观察到的选择行为与神经活动的测量结合起来，以确定后者是否与概率信念相关。其基本原理是，中脑中的多巴胺能活动——与强化学习理论一致——被认为编码了奖励预测错误，这可以通过功能性磁共振成像在纹状体中观察到。如果这种神经测量可以被验证为概率信念的代理，它可以用于对不可观察信念的推断。这进一步证明了受试者的决定在多大程度上依赖于当前信息和先前经验，这取决于这两种信息来源的特点。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Decision makers often consider information from a variety of sources, but since processing information is effortful not all information is always taken into account. This project investigates the contributions of two major sources of information to decision-making: any available information on the reward that an action will produce, and the history of rewards obtained from taking this action in the past. The extent to which decision makers rely on either of these sources of information is fundamental to understanding commonly observed choice patterns that economize on learning: Present information is often (at least partially) ignored and substituted for by relying on prior experience. Recent advances in economics model this trade-off, but do not typically provide an account of prior experience. Neuroscience and artificial intelligence, however, provide an account of learning from prior experience which is relevant to the formation of habits, for instance. This project aims to combine neuroscientific and economic accounts in one coherent theoretical framework, and to quantify experimentally the extent to which decisions rely on present information and prior experience, respectively. Such a measurement is crucial for understanding the circumstances under which a decision maker reassesses a recurring decision and breaks a habit. It will suggest policies that help incentivize decision makers to make deliberate decisions rather than following persistent habits. This is of great importance given the central role habits play in anxiety disorders, addiction, and arguably also in consumption decisions. The relative contributions of present information and prior experience are assessed in an experiment that is firmly rooted in the economic theory of rational inattention as well as computational models of reinforcement learning from cognitive neuroscience. The experiment combines observed choice behavior with a measurement of neural activity, in order to establish whether the latter is correlated with probabilistic beliefs. The rationale is that dopaminergic activity in the midbrain is – consistent with reinforcement learning theory – believed to encode a reward prediction error, which can be observed in the striatum using functional magnetic resonance imaging. If this neural measurement can be validated as a proxy for probabilistic beliefs, it could be leveraged for inference on unobservable beliefs. This provides further evidence on the extent to which subjects’ decisions rely on present information and prior experience, depending on the characteristics of these two sources of information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.