Multiple Time Scale Reinforcement Learning

该项目将研究一种新的方法来学习动态系统的模型，用于先进的强化学习架构。 它将开发一种方法，通过该方法，强化学习系统可以学习多个时间尺度模型，并将其用作分层学习和规划的基础。 该项目的目标是发展这种方法的数学理论，研究它与控制理论以及学习的行为和神经模型的关系，并在一些模拟学习任务中证明它的有效性。该项目对工程的意义在于，TD模型的使用将是RL架构的主要推广，使其具有更广泛的适用性。 它也有可能建立TD模型的实用性系统辨识在更传统的自适应控制。 该项目还对我们理解动物学习具有意义，能够以数学原理的方式模拟间接和直接关联及其相互作用。 这项研究的一个额外影响将是加强工程学、人工智能和生物学学习研究之间的联系，从而通过促进概念和方法的转移，为所有三个领域做出贡献。

This project will investigate a new approach to learning models of dynamical systems for use in advanced reinforcement learning architectures. It will develop a method by which a reinforcement learning system can learn multiple time scale models and use them as the basis for hierarchical learning and planning. the project(s objectives are to develop the mathematical theory of this approach, to examine its relationship to control theory and to behavioral and neural models of learning, and to demonstrate its effectiveness in a number of simulated learning tasks. The significance of the project for engineeirng is that the use of TD models will be major generalization of RL architectures, making them much more widely applicable. It also has the possibility of establshing the utility of TD modesl for system identification in more conventional adaptive control. The project also has implication for our understanding of animal learning able to model indirect and direct associations and their interactions in a mathematically principled way. An Additinoal impact of this research will be t o strengthen links between engineering, artifical intellegence, and biological studies of learning, thereby contributng to all three areas by facilitating a transfer of concepts and methods.