Fast Reinforcement Learning Using Multiple Models and State Decomposition

该项目试图为强化学习和近似动态规划（RLADP）开发更好的方法，以便能够处理在时间和空间上都具有更大复杂性的决策任务。强化学习系统是一种可以学习最大化任何性能或满意度的系统，基于他们观察环境的经验，对环境采取行动，并接收关于性能的反馈，类似于用于加强动物行为的痛苦或快乐。当前的强化学习方法在空间或时间上过于复杂时，学习速度不够快，无法表现良好。这个项目将开发新的方法来处理这种复杂性。该团队还将与IBM研究部门合作，并将尝试解决一个涉及插电式混合动力汽车车队管理的试验台问题。时间上的复杂性将通过使用多模型方法来处理，通过评估和更新标志着状态空间不同区域之间转换的地标状态来连接各种选项或技能。这类似于先前在PI的学习和自适应系统研讨会上提出的决策块和修改的Bellman方程的工作，但在其他方面是一个独特的，新的重要方向。空间的复杂性是由多智能体的方法，基于一种空间分解。

This project attempts to develop better methods for Reinforcement Learning and Approximate Dynamic Programming (RLADP), in order to be able to handle decision tasks with greater complexity both in time and in space. Reinforcement learning systems are systems which can learn to maximize any measure of performance or satisfaction, based on their experience of observing their environment, acting on the environment, and receiving feedback on performance, similar to the pain or pleasure which is used to reinforce animal behavior. Current reinforcement learning methods do not learn fast enough to perform well, when their environment is too complex in space or in time. This project will develop new methods to handle that kind of complexity. The team will also have a collaboration with IBM research, and will try to address a testbed problem involving the management of a fleet of plug-in hybrid cars.Complexity in time will be handled by use of a multiple model approach, connecting various options or skills by evaluation and updating of the landmark states which mark transitions between different regions of state space. This is similar to previous work on decision blocks and modified Bellman equations previously presented at the PI's workshop on learning and adaptive systems, but otherwise is a unique, new an important direction. Complexity in space is addressed by a multiagent approach, based on a kind of spatial decomposition.