Optimizing the depth and the direction of prospective planning using information values
Optimizing the depth and the direction of prospective planning using information values
复制标题
利用信息值优化前瞻性规划的深度和方向
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
M. Keramati
中科院分区：
文献类型：
--
作者：
Can Eren Sezener;A. Dezfouli;M. Keramati
通过模拟一棵通向未来的心理搜索树，可以评估行动的未来后果。然而，扩展深树在计算上是一项繁重的工作。因此，机器和人类使用一种直到习惯的计划方案，该方案模拟直到有限深度的环境，然后利用习惯性值作为未来可能出现的后果的代理。该方案中的两个悬而未决的问题是“搜索树应该向哪个方向扩展？”和“扩展应该何时停止？”在这里，我们提出了一个基于速度/精度权衡的原则性解决方案：在适当方向上进行更深层次的扩展会导致更准确的规划，但代价是决策速度较慢。仿真结果表明，该算法在网格环境下有效地扩展了搜索树。我们进一步证明，我们的算法可以解释动物和人类的几种行为模式，即时间压力对规划深度的影响，奖励大小对规划方向的影响，以及在训练过程中从目标导向行为到习惯性行为的逐渐转变。该算法还提供了几个在动物/人体实验中可验证的预测。
Evaluating the future consequences of actions is achievable by simulating a mental search tree into the future. Expanding deep trees, however, is computationally taxing. Therefore, machines and humans use a plan-until-habit scheme that simulates the environment up to a limited depth and then exploits habitual values as proxies for consequences that may arise in the future. Two outstanding questions in this scheme are “in which directions the search tree should be expanded?”, and “when should the expansion stop?”. Here we propose a principled solution to these questions based on a speed/accuracy tradeoff: deeper expansion in the appropriate directions leads to more accurate planning, but at the cost of slower decision-making. Our simulation results show how this algorithm expands the search tree effectively and efficiently in a grid-world environment. We further show that our algorithm can explain several behavioral patterns in animals and humans, namely the effect of time-pressure on the depth of planning, the effect of reward magnitudes on the direction of planning, and the gradual shift from goal-directed to habitual behavior over the course of training. The algorithm also provides several predictions testable in animal/human experiments.
DOI：
10.1037/0097-7403.30.2.104
发表时间：
2004-04-01
期刊：
JOURNAL OF EXPERIMENTAL PSYCHOLOGY-ANIMAL BEHAVIORAL PROCESSES
影响因子：
--
作者：
Holland, PC
通讯作者：
Holland, PC