Adaptive dynamic programming for uncertain nonlinear systems through coupling of nonlinear analysis and data-based learning

最优控制方法提供了一种将用户定义的成本与控制动作或决策相关联的方法。这些方法在广泛的应用领域产生了广泛的影响。汽车工业向自动化的转变，例如用计算机控制的电子控制系统取代机械系统，已经导致了燃料喷射、制动、节流等方面的效率。对于电动车辆，燃料经济性、驾驶性能和排放控制是发动机设计和控制策略的功能。对于机器人系统来说，实现最佳行为的愿望对于高效执行任务至关重要。同样，航空航天系统一直是最优控制方法的主流应用领域，因为性能和能源/燃料成本之间一直存在（并且将永远存在）紧密耦合。随着能源成本的上升和人们对能源生产对环境影响的认识的提高，最优控制现在可能在更广泛的应用领域中发挥及时的作用。美国能源部的能源效率和可再生能源办公室表示，通过优化工业系统，可以节省多达10- 20%的美国能源使用。不言而喻，最优控制解决方案可以在广泛的行业中产生重大影响，但真实的工程系统的最优控制解决方案的开发受到众多技术障碍的限制。最根本的和开放式的问题产生的障碍与发展中国家的最佳解决方案的存在的不确定性。该项目的主要问题是如何在获取系统知识的同时为不确定和复杂的工程系统做出最优控制决策。该项目的技术目标是由初步努力的假设和观察所激发的，即非线性分析方法可以用来设计实时近似最优解，同时并行后台处理方法可用于更新最优控制近似以提高性能。在这个项目中的智能优点是通过开发具有相关稳定性分析的闭环控制器类和先进的函数逼近方法来实现的，这些方法在学习近似最优控制解决方案的同时确保了对系统响应的充分探索。这项研究的结果将允许在更广泛的应用领域，系统表现出非线性行为和不确定性的最优控制实施。该框架的广泛影响是方法的合并，弥合了计算智能社区和控制系统社区之间的差距，使基于数据的学习方法能够优化控制性能。

Optimal control methods provide a means to associate a user-defined cost with control actions or decisions. These methods have made pervasive impacts in a wide class of application domains. The shift towards autonomy by the automotive industry in examples such as replacing mechanical systems with computer controlled electronic control systems, has resulted in efficiencies in fuel injection, braking, throttling, etc. For electric vehicles, fuel economy, drivability, and emission control are functions of the engine design and the control strategies. For robotic systems, the desire to achieve optimal behavior is essential for efficient task execution. Likewise, aerospace systems have always been a mainstream application domain for optimal control methods because there has always been (and always will be) a tight coupling between performance and energy/fuel costs. As the cost of energy and the awareness of the environmental impacts of producing energy have risen, optimal control may now play a timely role in a broader spectrum of application domains. The Energy Efficiency and Renewable Energy office of the U.S. Department of Energy indicates that as much as 10-20 percent of American energy use could be saved by optimizing industrial systems. It is self-evident that optimal control solutions can have significant impacts in a wide range of industries, but the development of optimal control solutions for real engineering systems is limited by numerous technical barriers. The most fundamental and open-ended problems arise from barriers associated with developing optimal solutions in the presence of uncertainty. The driving question in this project is how to arbitrate between gaining knowledge about a system while simultaneously making the optimal control decision for engineering systems that are uncertain and complex.The technical aims of this project are motivated by the hypothesis and observations from preliminary efforts that nonlinear analysis methods can be exploited to design real-time approximate optimal solutions, while concurrent background processing methods can be used to update the optimal control approximation for improved performance. Intellectual merits in this project are realized through the development of classes of closed-loop controllers with associated stability analysis, and advanced function approximation methods, that ensure sufficient exploration of the system response while learning the approximate optimal control solution. Outcomes of this research would allow for optimal control implementation in a broader class of application domains where the system exhibits nonlinear behaviors and uncertainty. The broad impact of this framework is a merger of methods that bridge the gap between the computational intelligence community and control systems community to enable data-based learning methods to optimize control performance.