Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration
Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration
复制标题
DOI：
10.1098/rstb.2007.2098
复制
发表时间：
2007-05-29
影响因子：
6.3
通讯作者：
Yu, Angela J.
中科院分区：
文献类型：
--
作者：
Cohen, Jonathan D.;McClure, Samuel M.;Yu, Angela J.
我们在日常生活中所做的许多大大小小的决定--选择哪种冰淇淋，从事什么研究项目，与哪个伴侣结婚--都需要在承诺和利用特定选择的好处之前探索替代品。此外，面对不断变化的需求或情况，许多决定需要重新评估，并进一步探索替代办法。也就是说，我们的决策通常取决于更高层次的选择：是利用众所周知但可能不太理想的替代品，还是探索风险较大但可能更有利可图的替代品。自适应代理如何在开发和探索之间进行选择仍然是一个重要而开放的问题，在行为科学和脑科学中受到的关注相对有限。这种选择可能取决于若干因素，包括对环境的熟悉程度、环境可能变化的速度以及利用已知回报来源与通过探索减少不确定性的成本的相对价值。对于勘探与开采问题，没有已知的一般最佳解决方案，并且对于一般情况的解决方案可能确实是不可能的。然而，在约束条件下的最优策略已经有了正式的分析。也有一些具体的建议，人类和动物在特定的实验条件下如何应对这个问题，以及有关大脑机制的建议。在这里，我们简要回顾了这项工作，讨论了探索和开发如何在大脑中进行介导，并强调了一些有前途的未来研究方向。
Many large and small decisions we make in our daily lives - which ice cream to choose, what research projects to pursue, which partner to marry - require an exploration of alternatives before committing to and exploiting the benefits of a particular choice. Furthermore, many decisions require re-evaluation, and further exploration of alternatives, in the face of changing needs or circumstances. That is, often our decisions depend on a higher level choice: whether to exploit well known but possibly suboptimal alternatives or to explore risky but potentially more profitable ones. How adaptive agents choose between exploitation and exploration remains an important and open question that has received relatively limited attention in the behavioural and brain sciences. The choice could depend on a number of factors, including the familiarity of the environment, how quickly the environment is likely to change and the relative value of exploiting known sources of reward versus the cost of reducing uncertainty through exploration. There is no known generally optimal solution to the exploration versus exploitation problem, and a solution to the general case may indeed not be possible. However, there have been formal analyses of the optimal policy under constrained circumstances. There have also been specific suggestions of how humans and animals may respond to this problem under particular experimental conditions as well as proposals about the brain mechanisms involved. Here, we provide a brief review of this work, discuss how exploration and exploitation may be mediated in the brain and highlight some promising future directions for research.