The Generality and Practicality of Reinforcement Learning for Automatic Control

点击翻译按钮获取中文摘要

Recent discoveries of the theoretical relationships between reinforcement learning and dynamic programming suggest exciting possibilities for developing automatic controllers that learn with experience to follow optimal control strategies. Combining reinforcement learning algorithms with the adaptive structure that neural networks provide results in theoretically optimal controllers that have more flexibility, and thus are more general, than current adaptive control techniques. However, for reinforcement learning networks to be practical, the efficiency with which they learn must be improved. In previous work, the PI identified one cause of slow learning to be difficulty of discovering useful features by the hidden units of the network. This difficulty has also been recognized within the supervisedlearning paradigm and a number of alternatives to the common error back propagation algorithm have been shown to significantly reduce learning time. These ideas will be extended to the reinforcement learning paradigm and their potential for reducing the learning time of reinforcement based networks will be explored. The objective is to alleviate the problem of training hidden units and to identify any remaining limitations of reinforcement learning networks that restrict their generality and practicality as real time control techniques. The methods will include both simulation studies and implementations as controllers of physical systems.