Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective.
Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective.
复制标题
DOI：
10.1016/j.cognition.2008.08.011
复制
发表时间：
2009-12
期刊：
影响因子：
3.4
通讯作者：
Barto AG
中科院分区：
文献类型：
--
作者：
Botvinick MM;Niv Y;Barto AG
长期以来，人类和动物行为的研究一直强调其层次结构--将正在进行的行为分割成离散的任务，这些任务由子任务序列组成，而子任务序列又由简单的动作组成。行为的层次结构也一直是神经科学的兴趣所在，它被广泛认为反映了前额叶皮层的功能。在本文中，我们重新审视行为层次结构和它的神经基板的角度在计算强化学习的最新发展。具体来说，我们考虑了一组方法统称为分层强化学习，它通过允许学习代理将动作聚合到可重用的子程序或技能中来扩展强化学习范式。仔细研究分层强化学习的组成部分，可以发现它们如何映射到神经结构上，特别是背外侧和眶前额叶皮层内的区域。它还提出了层次强化学习可能为现有的层次结构行为心理模型提供补充的具体方法。分层强化学习带来的一个特别重要的问题是，学习如何识别新的动作例程，这些动作例程可能为解决未来广泛的问题提供有用的构建模块。在这里和其他许多方面，分层强化学习为研究分层结构行为的计算和神经基础提供了一个有吸引力的框架。
Research on human and animal behavior has long emphasized its hierarchical structure — the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.
登录
查看更多内容
影响因子：
5.3
作者：
Averbeck, Bruno B.;Lee, Daeyeol
通讯作者：
Lee, Daeyeol
影响因子：
16.2
作者：
Bor, D;Duncan, J;Owen, AM
通讯作者：
Owen, AM
影响因子：
2.1
作者：
Aldridge, JW;Berridge, KC;Rosen, AR
通讯作者：
Rosen, AR
DOI：
10.1109/tsmc.1983.6313077
发表时间：
1983-01-01
期刊：
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS
影响因子：
--
作者：
BARTO, AG;SUTTON, RS;ANDERSON, CW
通讯作者：
ANDERSON, CW
影响因子：
5.4
作者：
Anderson, JR;Bothell, D;Qin, YL
通讯作者：
Qin, YL