The Generality and Practicality of Reinforcement Learning for Automatic Control

最近发现的理论关系， 强化学习和动态编程提出了令人兴奋的 开发自动控制器的可能性， 最佳控制策略的经验。 结合 具有自适应结构的强化学习算法， 神经网络提供理论上最优的结果 具有更大灵活性的控制器，因此更通用， 目前的自适应控制技术。 但对于 强化学习网络要实用，效率高 他们的学习能力必须得到提高。在以前的工作中，PI 我发现学习缓慢的一个原因是 通过网络的隐藏单元发现有用的特征。 这一困难也得到了国际社会的承认。 监督学习范式和一些替代方案， 常见的误差反向传播算法已被证明 大大缩短了学习时间。 这些想法将被延伸 强化学习范式及其潜力， 减少基于强化的网络的学习时间将是 探讨了 目的是缓解培训问题 隐藏的单位，并确定任何剩余的限制， 强化学习网络限制了其通用性， 实用性作为真实的时间控制技术。 述方法将 包括仿真研究和作为控制器实现 物理系统。

Recent discoveries of the theoretical relationships between reinforcement learning and dynamic programming suggest exciting possibilities for developing automatic controllers that learn with experience to follow optimal control strategies. Combining reinforcement learning algorithms with the adaptive structure that neural networks provide results in theoretically optimal controllers that have more flexibility, and thus are more general, than current adaptive control techniques. However, for reinforcement learning networks to be practical, the efficiency with which they learn must be improved. In previous work, the PI identified one cause of slow learning to be difficulty of discovering useful features by the hidden units of the network. This difficulty has also been recognized within the supervisedlearning paradigm and a number of alternatives to the common error back propagation algorithm have been shown to significantly reduce learning time. These ideas will be extended to the reinforcement learning paradigm and their potential for reducing the learning time of reinforcement based networks will be explored. The objective is to alleviate the problem of training hidden units and to identify any remaining limitations of reinforcement learning networks that restrict their generality and practicality as real time control techniques. The methods will include both simulation studies and implementations as controllers of physical systems.