CRII: CPS: A Self-Learning Intelligent Control Framework for Networked Cyber-Physical Systems

该项目的目标是解决相互作用的智能物理系统在机器学习方面的挑战。该方法是探索强化学习（RL）策略，其中系统在行为正确时获得奖励，当与之交互的系统可能以不一致的方式反应时，与物理系统交互。研究结果有望为一个新的自学习智能控制框架做出贡献，在这个框架中，设计中的系统可以决定如何与不一致的邻居进行交互，从而改善它们的学习方式。这将推进网络网络物理系统（CPS）的强化学习，这些系统在相互作用（例如，无人机系统）时可能会出现紧急行为，并且由于其分布式性质的不确定性（例如，智能电网）而经常不一致。预计他将对科学领域做出三大基础性贡献。首先，一种新的分布式强化学习算法将在不需要外部监督的情况下自动学习合适的奖励函数。这项工作将放松人类的努力，并将强化学习算法扩展到更复杂的环境中。其次，新的基于迁移学习的强化学习架构将通过重用来自多个来源的过去知识来设计。本设计将进一步加快网络化CPS的学习进程。第三，将该方法应用于多机器人实验平台，以促进机器人学习的应用。推广和传播计划培养K-12学生和来自代表性不足群体的学生的科学好奇心，激发他们对科学、技术、工程和数学（STEM）项目的兴趣。此外，将该项目的前沿研究成果整合到新课程中，将有助于留住现有的STEM学生。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project is to addresses challenges in machine learning for intelligent physical systems that interact with one another. The approach is to explore Reinforcement Learning (RL) strategies, where systems are rewarded when behaving correctly, for interacting physical systems when the systems with which they interact may react in inconsistent ways. The results are expected to contribute to a new self-learning intelligent control framework, where the systems under design can decide how to interact with their inconsistent neighbors in a way that will improve how they learn. This will advance reinforcement learning for networked cyber-physical systems (CPS) which can have emergent behaviors when they interact (for example, unmanned aerial systems) and are frequently inconsistent due to uncertainties in their distributed nature (for example, the smart grid).Three major fundamental contributions to the scientific field are expected. First, a new distributed RL algorithm will learn suitable reward functions automatically without requiring external supervisions. This work will relax human efforts and scale RL algorithms to more complex environment. Second, novel transfer learning-based RL architectures will be designed by reusing past knowledge from multiple sources. This design will further accelerate learning process in networked CPS. Third, this proposed method will be implemented on a multi-robot testbed to advance the learning in robot applications. Outreach and dissemination plans cultivate the scientific curiosity of K-12 students, and students from underrepresented groups, and motivate their interests in Science, Technology, Engineering, and Math (STEM) programs. Furthermore, the integration of the project's cutting-edge research results into new courses will aid retention of current STEM students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.