ERI: Improving the Learning Efficiency of Adaptive Optimal Control Systems in Information-Limited Environments

这项工程研究启动(ERI)拨款将资助研究，使在不确定环境中运行的复杂工程系统能够高效地实时学习最佳控制策略，并应用于联网和自动驾驶车辆，从而促进科学进步和促进国家繁荣和福利。人工智能、汽车、机器人和能源领域的许多新兴控制系统要求在缺乏详细系统知识的情况下，基于有限的输入数据在组件网络中识别和执行最佳动作。基于学习的方法已经被开发出来以满足这一要求，但面临着学习速度非常慢、对用于启动学习过程的控制策略的限制性要求以及由于传感器的限制和单个组件之间稀疏的数据共享而导致的复杂性的挑战。该项目将通过建立一个新的学习效率高的控制框架来克服这些挑战，该框架整合了现有方法的优点，并展示了处理丢失数据流和优化系统组件之间的通信结构的新解决方案。当应用于复杂交通场景中的自动驾驶车辆网络时，该框架可能会提高道路安全并减少道路死亡。这个项目的更广泛的影响包括向公众展示如何安全地利用人工智能和自动控制，以及培训和准备本科生和研究生继续深造和进修STEM职业。本研究旨在为开发具有完全未知系统模型和部分可观测性条件下的非线性动态系统的学习效率、自适应最优控制框架做出基本贡献，并使该框架能够应用于具有非平凡通信拓扑的网络控制系统。它将通过发展一种新的强化学习的混合迭代形式来实现这一结果，该形式即使在系统模型和初始允许的控制策略不可用的情况下也能获得二次收敛速度。受分层强化学习思想的启发，将创建一种两层学习高效的方法，以实现单个网络代理的稳健分布式控制策略和最优网络通信拓扑的同时学习，包括在代理之间存在通信延迟的情况下。微交通模拟和物理实验将被用来在几个编队控制场景中测试避碰背景下的理论框架。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Engineering Research Initiation (ERI) grant will fund research that enables efficient, on-the-fly learning of optimal control strategies for complex engineering systems operating in uncertain environments, with application to connected and autonomous vehicles, thereby promoting the progress of science and advancing the national prosperity and welfare. Many emerging control systems in the artificial intelligence, automotive, robotic, and energy fields require that optimal actions be identified and executed across a network of components in the absence of detailed system knowledge and based on limited input data. Learning-based approaches have been developed to meet this requirement, but are challenged by very slow rates of learning, restrictive requirements on the control policy used to initiate the learning process, and complications due to sensor limitations and sparse data sharing between individual components. This project will overcome these challenges by building a new learning-efficient control framework that integrates advantages of existing methods and demonstrates new solutions for handling missing data streams and optimizing the communication structure between system components. When applied to networks of autonomous vehicles in complex traffic scenarios, the framework may enable improvements in roadway safety and reduction in road fatalities. The broader impacts of this project include outreach efforts to the public intended to show how artificial intelligence and automatic control can be safely leveraged, as well as training and preparation of undergraduate and graduate students to pursue further education and advanced STEM careers.This research aims to make fundamental contributions to the development of a learning-efficient, adaptive optimal control framework for nonlinear dynamical systems with completely unknown system models and under conditions of partial observability, and to enable the application of this framework to networked control systems with nontrivial communication topologies. It will achieve this outcome by developing a new hybrid iterative form of reinforcement learning that achieves a quadratic rate of convergence even if a system model and an initial admissible control policy are unavailable. Inspired by ideas from hierarchical reinforcement learning, a two-layer learning-efficient method will be created to enable simultaneous learning of robust distributed control strategies for individual network agents and an optimal network communication topology, including in the presence of communication delays between agents. Micro-traffic simulations and physical experiments will be used to test the theoretical framework in the context of collision avoidance in several formation control scenarios.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.