Collaborative Research: Fast reinforcement learning using multiple models and state decompositions for apllications to Plug-in Hybrid Vehicles

协作研究：使用多个模型和状态分解的快速强化学习在插电式混合动力汽车中的应用

• 批准号: 1408279
• 负责人: Kumpati Narendra
• 金额: $ 20万
• 依托单位: Yale University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-15 至 2017-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1408279&HistoricalAwards=false
• 关键词: Collaborative; Research; Fast; reinforcement; learning

Learning schemes in general, are known for their slow convergence rates. The proposal suggests various modifications for improving the speed of learning. The first is the use of multiple models which has proved very effective in other areas of systems theory. A second modification is the use of decentralized or partially decentralized multi-agent state decomposition approaches. Both of the above will make learning schemes in general and reinforcement schemes in particular fast converging and applicable to a wide class of real-world problems. The proposed research will develop the foundations of fast reinforcement learning using these two broad approaches, and will also apply these methods to the optimal control of a fleet of Plug-in Hybrid Electric Vehicles. The outcome of this research will have a societal impact for optimally controlling a fleet of Plug-in Hybrid Electric Vehicles for energy efficiency. The application of the proposed research will have extended application in other areas of systems theory such as neurobiology. Graduate and undergraduate students will be trained in learning schemes for multidisciplinary applications. Cross-fertilization of ideas will be facilitated through a bi-annual workshop on adaptive and learning systems and by leveraging a multidisciplinary Institute for Mathematical Modeling and Computational Science (iMMCS).The proposal focuses on improving convergence speed of learning schemes in multiple agents in order to overcome the limitations of dimensionality, as well as in situations where infrequent communication exists between agents. Innovation is in the use of multiple models and state decomposition to speed up the learning and convergence. The technical approach lies in using multiple identification models; reinforcement learning by using decentralized or partially decentralized multi-agent state decomposition approaches; and evaluating fast reinforcement learning in an application test-bed for controlling a fleet of Plug-in Hybrid Electric Vehicles (PHEVs). The Research Team further plans to quantify the trade-off between learning speed and quality of the "learned solutions".

一般来说，学习方案以其缓慢的收敛速度而闻名。 该提案提出了各种修改，以提高学习速度。第一个是使用多模型，这在系统理论的其他领域已经证明是非常有效的。第二个修改是使用分散或部分分散的多智能体状态分解方法。 上述两种方法都将使学习方案特别是强化方案快速收敛，并适用于广泛的现实问题。拟议的研究将使用这两种广泛的方法开发快速强化学习的基础，并将这些方法应用于插电式混合动力电动汽车车队的最佳控制。这项研究的结果将对优化控制插电式混合动力电动汽车的能源效率产生社会影响。这项研究的应用将扩展到系统理论的其他领域，如神经生物学。研究生和本科生将接受多学科应用学习计划的培训。 通过一年两次的自适应和学习系统研讨会，并利用多学科的数学建模和计算科学研究所（iMMCS），将促进思想的相互交流。该提案的重点是提高多个智能体中学习方案的收敛速度，以克服维度的限制，以及智能体之间存在不频繁通信的情况。 创新之处在于采用多模型和状态分解来加速学习和收敛。技术方法在于使用多个识别模型;通过使用分散或部分分散的多智能体状态分解方法进行强化学习;以及在用于控制插电式混合动力电动汽车（PHEV）车队的应用测试台中评估快速强化学习。 研究小组进一步计划量化学习速度和“学习解决方案”质量之间的权衡。