Neural representation of belief states during decision-making under uncertainty

知道环境的哪些方面与我们的目标相关对决策至关重要，但也为处理感官输入提供了重要的背景。在红绿灯前等待时，行人会无视周围汽车的制造商和颜色，只有在绿灯时才开始过马路。然而，在等待出租车时，过往车辆的颜色和型号是决定何时放弃手臂的最重要因素。但是，我们环境中的感官信息通常也是嘈杂的，关于所需状态(例如，灯是绿的)是否正确的知识可能不确定。因此，在现实环境中进行决策需要我们以反映当前目标和背景的方式处理感官输入，但也要考虑到潜在的不确定性。在这个项目中，我们将研究决策过程中关于不同可能感知和可能背景的不确定性是如何反映在人脑中的。决策和强化学习的形式理论表明，环境不同状态的表现方式对决策有重要影响(Sutton&Barto，1998)。他们建议，在面临不确定性的情况下，将有关可能状态集的信息与这些状态中的每一种状态当前为真的确定性结合起来可能是最佳的。这些所谓的“信念状态”(Kaelling等人，1996)在决策的理论工作中发挥了重要作用，但对大脑中是否存在相应的表征知之甚少。在我们之前的工作(Schuck，2015,2016；Kaplan，Schuck&Doeller，2017)的基础上，我们提出，大脑中的概率信念状态表征反映在一个整合的分布式神经代码中，其中状态认同和这些状态为真的概率是多路复用的，并且这些表征可以在腹内侧前额叶皮质中找到。为了验证我们的假设，我们引入了一个新的多步骤决策任务，其中以前步骤的事件和不确定性为当前步骤提供了上下文。使用贝叶斯分类方法，我们将检查每个步骤(例如van Bergen等人，2015)支持不同状态的相对多变量fMRI证据，并测试状态上的神经编码分布是否对应于信念状态模型的预测。我们的项目将允许我们以计算简洁的方式全面研究神经表示法，这些表示法以计算简洁的方式对决策提供不确定性分级的上下文影响。

Knowing which aspects of the environment are relevant for our goal is crucial for decision making, but also provides important context for processing sensory input. Waiting at a traffic light, a pedestrian will disregard the manufacturers and colors of the cars around him and only begin to cross the road when the light is green. When waiting for a taxi, however, a passing car’s color and model are most important for deciding when to waive the arm. But sensory information in our environment is often also noisy, and knowledge about whether a desired state (e.g., ‘the light is green’) is true or not can be uncertain. Decision-making with in realistic environments therefore requires us to process sensory input in ways that reflect the current goal and context, but also take potential uncertainty into account. In this project, we will investigate how uncertainty regarding different possible percepts and possible contexts during decision making is reflected in the human brain. Formal theories of decision making and reinforcement learning have shown that the way different states of the environment are represented has important implications for decision making (Sutton & Barto, 1998). They suggest that in the face of uncertainty, it may be optimal to integrate information about the set of possible states with the certainty that each of these states is currently true. These so-called “belief states” (Kaelbling et al., 1996) have played an important role in theoretical work on decision making, but little is known about whether corresponding representations exist in the brain. Drawing on our previous work (Schuck, 2015, 2016; Kaplan, Schuck & Doeller, 2017), we propose that probabilistic belief state representations in the brain are reflected in an integrative and distributed neural code in which state identities and the probabilities that these states are true are multiplexed, and that these representations can be found in the ventromedial prefrontal cortex. To test our hypothesis, we introduce a novel multi-step decision making task in which the events and uncertainty of previous steps provide the context for the current step. Using Bayesian classification approaches, we will examine the relative multivariate fMRI evidence in favor of different states at each step (e.g., van Bergen et al., 2015) and test whether the neurally encoded distribution over states corresponds to the predictions of a belief state model. Our project will allow us to comprehensively investigate the neural representations that provide an uncertainty-graded contextual influence on decision making in a computationally concise way.