Active Perception and Reinforcement Learning
Active Perception and Reinforcement Learning
复制标题
DOI：
10.1162/neco.1990.2.4.409
复制
发表时间：
1990-06
影响因子：
2.9
通讯作者：
S. Whitehead;D. Ballard
中科院分区：
文献类型：
--
作者：
S. Whitehead;D. Ballard
本文考虑自适应控制架构，集成主动感觉运动系统与决策系统的基础上强化学习。主动感知的一个不可避免的后果是，主体的内部表征经常混淆外部世界的状态。我们称这种现象为感知混淆，并表明它会使现有的强化学习算法在最优决策策略方面不稳定。一个新的决策系统，克服了这些困难。该系统在整个决策周期内包含一个感知子周期，并使用修改后的学习算法来抑制感知混叠的影响。其结果是一个控制架构，不仅学习如何解决一个任务，而且还在哪里集中注意力，以收集必要的感官信息。
This paper considers adaptive control architectures that integrate active sensorimotor systems with decision systems based on reinforcement learning. One unavoidable consequence of active perception is that the agent's internal representation often confounds external world states. We call this phenomenon perceptual aliasing and show that it destabilizes existing reinforcement learning algorithms with respect to the optimal decision policy. A new decision system that overcomes these difficulties is described. The system incorporates a perceptual subcycle within the overall decision cycle and uses a modified learning algorithm to suppress the effects of perceptual aliasing. The result is a control architecture that learns not only how to solve a task but also where to focus its attention in order to collect necessary sensory information.