Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys.
Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys.
复制标题
DOI：
10.3758/s13415-022-01059-z
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Soltani A
中科院分区：
文献类型：
--
作者：
Woo JH;Aguirre CG;Bari BA;Tsutsui KI;Grabenhorst F;Cohen JY;Schultz W;Izquierdo A;Soltani A
尽管奖励环境是不可预测和不确定的，但通常表现出一定的规律，在这些环境中导航的动物试图发现并利用这种规律来调整自己的行为。然而，成功的学习要求动物也适应与这些规律相关的不确定性。在这里，我们分析了来自小鼠和猴子两个可比较的动态觅食任务的选择数据，以研究对不同类型不确定性的调整机制。在这些任务中，动物在两种概率奖励的选择中做出选择，而基线奖励概率在没有任何提示的情况下进行可变次数的试验后发生变化。为了衡量行为的调整，我们应用了基于信息理论的多个指标来量化行为的一致性，并使用强化学习模型拟合选择数据。我们发现，在这两个物种中，学习和选择都受到奖励结果的不确定性（就确定更好的选择而言）和对环境何时可能变化的预期的影响。然而，这些影响是通过不同的机制介导的。首先，对更好的选择的更多不确定性导致老鼠学习和遗忘速度变慢，而对猴子没有显著影响。其次，在小鼠中，对区块开关的预期伴随着更慢的学习、更快的遗忘和选择随机性的增加，而在猴子中，它只降低了学习率。总的来说，在证明基于信息理论的指标在检查适应性行为方面的有用性的同时，我们的研究为根据奖励环境的不确定性在学习和选择行为中进行多种类型的调整提供了证据。
Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.