Adaptive dynamic programming for uncertain nonlinear systems through coupling of nonlinear analysis and data-based learning

最优控制方法提供了一种将用户定义的成本与控制行动或决策相关联的方法。这些方法在广泛的应用领域中产生了广泛的影响。汽车行业向自动驾驶的转变，例如用计算机控制的电子控制系统取代机械系统，提高了燃油喷射、制动、节流等方面的效率。对于电动汽车来说，燃油经济性、驾驶性和排放控制是发动机设计和控制策略的功能。对于机器人系统而言，实现最佳行为的愿望对于有效执行任务至关重要。同样，航空航天系统一直是最优控制方法的主流应用领域，因为性能与能源/燃料成本之间一直存在（并且将永远存在）紧密耦合。随着能源成本和对能源生产对环境影响的认识不断提高，最优控制现在可能在更广泛的应用领域中发挥及时的作用。美国能源部的能源效率和可再生能源办公室指出，通过优化工业系统，可以节省多达10- 20%的美国能源使用。不言而喻，最优控制解决方案可以在广泛的行业中产生重大影响，但实际工程系统的最优控制解决方案的发展受到许多技术障碍的限制。最基本和最开放的问题产生于在不确定性存在的情况下开发最佳解决方案的障碍。这个项目的驱动问题是如何在获得系统知识的同时，对不确定和复杂的工程系统做出最优控制决策。这个项目的技术目标是由假设和观察到的初步努力，非线性分析方法可以被用来设计实时近似最优解，而并发后台处理方法可以用来更新最优控制近似，以提高性能。本项目的智力优势是通过开发具有相关稳定性分析的闭环控制器类和先进的函数逼近方法来实现的，这些方法确保在学习近似最优控制解的同时充分探索系统响应。这项研究的结果将允许在系统表现出非线性行为和不确定性的更广泛的应用领域中实现最优控制。该框架的广泛影响是将计算智能社区和控制系统社区之间的差距结合起来，使基于数据的学习方法能够优化控制性能。

Optimal control methods provide a means to associate a user-defined cost with control actions or decisions. These methods have made pervasive impacts in a wide class of application domains. The shift towards autonomy by the automotive industry in examples such as replacing mechanical systems with computer controlled electronic control systems, has resulted in efficiencies in fuel injection, braking, throttling, etc. For electric vehicles, fuel economy, drivability, and emission control are functions of the engine design and the control strategies. For robotic systems, the desire to achieve optimal behavior is essential for efficient task execution. Likewise, aerospace systems have always been a mainstream application domain for optimal control methods because there has always been (and always will be) a tight coupling between performance and energy/fuel costs. As the cost of energy and the awareness of the environmental impacts of producing energy have risen, optimal control may now play a timely role in a broader spectrum of application domains. The Energy Efficiency and Renewable Energy office of the U.S. Department of Energy indicates that as much as 10-20 percent of American energy use could be saved by optimizing industrial systems. It is self-evident that optimal control solutions can have significant impacts in a wide range of industries, but the development of optimal control solutions for real engineering systems is limited by numerous technical barriers. The most fundamental and open-ended problems arise from barriers associated with developing optimal solutions in the presence of uncertainty. The driving question in this project is how to arbitrate between gaining knowledge about a system while simultaneously making the optimal control decision for engineering systems that are uncertain and complex.The technical aims of this project are motivated by the hypothesis and observations from preliminary efforts that nonlinear analysis methods can be exploited to design real-time approximate optimal solutions, while concurrent background processing methods can be used to update the optimal control approximation for improved performance. Intellectual merits in this project are realized through the development of classes of closed-loop controllers with associated stability analysis, and advanced function approximation methods, that ensure sufficient exploration of the system response while learning the approximate optimal control solution. Outcomes of this research would allow for optimal control implementation in a broader class of application domains where the system exhibits nonlinear behaviors and uncertainty. The broad impact of this framework is a merger of methods that bridge the gap between the computational intelligence community and control systems community to enable data-based learning methods to optimize control performance.