Doctoral Dissertation Research In DRMS: Reinforcement Learning and Attention in Decision Making

决策者经常考虑来自各种来源的信息，但由于处理信息是费力的，因此并非所有信息都被考虑在内。该项目调查了两个主要信息来源对决策的贡献：关于一个行动将产生的奖励的任何可用信息，以及过去采取该行动所获得的奖励的历史。决策者对这些信息来源的依赖程度对于理解通常观察到的节省学习的选择模式至关重要：现有信息通常（至少部分）被忽视，并被依赖于先前的经验所取代。经济学的最新进展为这种权衡建立了模型，但通常没有提供对先前经验的解释。然而，神经科学和人工智能提供了一种从先前经验中学习的解释，例如，这与习惯的形成有关。该项目旨在将联合收割机神经科学和经济账户结合在一个连贯的理论框架中，并通过实验量化决策分别依赖于当前信息和先前经验的程度。这种测量对于理解决策者重新评估重复性决策和打破习惯的情况至关重要。它将建议有助于激励决策者做出深思熟虑的决定，而不是遵循持久的习惯的政策。这一点非常重要，因为习惯在焦虑症、成瘾以及消费决策中扮演着核心角色。目前的信息和先前的经验的相对贡献进行评估的实验，是牢牢植根于经济理论的理性疏忽，以及从认知神经科学的强化学习的计算模型。该实验将观察到的选择行为与神经活动的测量相结合，以确定后者是否与概率信念相关。其基本原理是，中脑中的多巴胺能活动-与强化学习理论一致-被认为编码了奖励预测错误，这可以通过功能性磁共振成像在纹状体中观察到。如果这种神经测量可以被验证为概率信念的代理，那么它可以被用来推断不可观察的信念。这进一步证明了受试者的决定在多大程度上依赖于当前信息和先前经验，这取决于这两种信息来源的特征。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Decision makers often consider information from a variety of sources, but since processing information is effortful not all information is always taken into account. This project investigates the contributions of two major sources of information to decision-making: any available information on the reward that an action will produce, and the history of rewards obtained from taking this action in the past. The extent to which decision makers rely on either of these sources of information is fundamental to understanding commonly observed choice patterns that economize on learning: Present information is often (at least partially) ignored and substituted for by relying on prior experience. Recent advances in economics model this trade-off, but do not typically provide an account of prior experience. Neuroscience and artificial intelligence, however, provide an account of learning from prior experience which is relevant to the formation of habits, for instance. This project aims to combine neuroscientific and economic accounts in one coherent theoretical framework, and to quantify experimentally the extent to which decisions rely on present information and prior experience, respectively. Such a measurement is crucial for understanding the circumstances under which a decision maker reassesses a recurring decision and breaks a habit. It will suggest policies that help incentivize decision makers to make deliberate decisions rather than following persistent habits. This is of great importance given the central role habits play in anxiety disorders, addiction, and arguably also in consumption decisions. The relative contributions of present information and prior experience are assessed in an experiment that is firmly rooted in the economic theory of rational inattention as well as computational models of reinforcement learning from cognitive neuroscience. The experiment combines observed choice behavior with a measurement of neural activity, in order to establish whether the latter is correlated with probabilistic beliefs. The rationale is that dopaminergic activity in the midbrain is – consistent with reinforcement learning theory – believed to encode a reward prediction error, which can be observed in the striatum using functional magnetic resonance imaging. If this neural measurement can be validated as a proxy for probabilistic beliefs, it could be leveraged for inference on unobservable beliefs. This provides further evidence on the extent to which subjects’ decisions rely on present information and prior experience, depending on the characteristics of these two sources of information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.