ERI: Improving the Learning Efficiency of Adaptive Optimal Control Systems in Information-Limited Environments

这项工程研究启动（ERI）赠款将资助研究，为在不确定环境中运行的复杂工程系统提供高效、实时的最优控制策略学习，并应用于联网和自动驾驶汽车，从而促进科学进步，促进国家繁荣和福利。在人工智能、汽车、机器人和能源领域，许多新兴的控制系统需要在缺乏详细系统知识和基于有限输入数据的情况下，在组件网络中识别和执行最佳操作。基于学习的方法已经被开发出来以满足这一要求，但面临着学习速度非常慢、对用于启动学习过程的控制策略的限制性要求以及由于传感器限制和单个组件之间稀疏数据共享造成的复杂性的挑战。该项目将通过构建一个新的学习效率控制框架来克服这些挑战，该框架集成了现有方法的优点，并展示了处理缺失数据流和优化系统组件之间通信结构的新解决方案。当应用于复杂交通场景中的自动驾驶汽车网络时，该框架可以改善道路安全性并减少道路死亡人数。该项目的更广泛影响包括向公众展示如何安全地利用人工智能和自动控制，以及培训和准备本科生和研究生继续接受教育和高级STEM职业。本研究旨在为具有完全未知系统模型和部分可观测条件的非线性动力系统开发一种学习效率高、自适应的最优控制框架，并使该框架能够应用于具有非平凡通信拓扑的网络控制系统。它将通过开发一种新的混合迭代形式的强化学习来实现这一结果，即使系统模型和初始可接受的控制策略不可用，也能实现二次收敛率。受分层强化学习思想的启发，将创建一种两层学习高效方法，以同时学习单个网络代理的鲁棒分布式控制策略和最佳网络通信拓扑，包括代理之间存在通信延迟。微交通模拟和物理实验将用于测试理论框架在避碰的背景下，在几个编队控制场景。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This Engineering Research Initiation (ERI) grant will fund research that enables efficient, on-the-fly learning of optimal control strategies for complex engineering systems operating in uncertain environments, with application to connected and autonomous vehicles, thereby promoting the progress of science and advancing the national prosperity and welfare. Many emerging control systems in the artificial intelligence, automotive, robotic, and energy fields require that optimal actions be identified and executed across a network of components in the absence of detailed system knowledge and based on limited input data. Learning-based approaches have been developed to meet this requirement, but are challenged by very slow rates of learning, restrictive requirements on the control policy used to initiate the learning process, and complications due to sensor limitations and sparse data sharing between individual components. This project will overcome these challenges by building a new learning-efficient control framework that integrates advantages of existing methods and demonstrates new solutions for handling missing data streams and optimizing the communication structure between system components. When applied to networks of autonomous vehicles in complex traffic scenarios, the framework may enable improvements in roadway safety and reduction in road fatalities. The broader impacts of this project include outreach efforts to the public intended to show how artificial intelligence and automatic control can be safely leveraged, as well as training and preparation of undergraduate and graduate students to pursue further education and advanced STEM careers.This research aims to make fundamental contributions to the development of a learning-efficient, adaptive optimal control framework for nonlinear dynamical systems with completely unknown system models and under conditions of partial observability, and to enable the application of this framework to networked control systems with nontrivial communication topologies. It will achieve this outcome by developing a new hybrid iterative form of reinforcement learning that achieves a quadratic rate of convergence even if a system model and an initial admissible control policy are unavailable. Inspired by ideas from hierarchical reinforcement learning, a two-layer learning-efficient method will be created to enable simultaneous learning of robust distributed control strategies for individual network agents and an optimal network communication topology, including in the presence of communication delays between agents. Micro-traffic simulations and physical experiments will be used to test the theoretical framework in the context of collision avoidance in several formation control scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.