CAREER: A Skill-Driven Cooperative Learning Framework for Cyber-Physical Autonomy

该项目研究了新的强化学习（RL）方法，用于网络物理自治，以弥合当前智能系统与人类智能之间的差距。许多信息物理系统（CPS）的本质是分布式的，异构的，高维的，使得手工编码的功能和特定于任务的信息很难设计的学习方案。通常需要大量的训练数据来实现所需的性能，但这限制了对其他任务的推广。因此，本项目旨在探索新的强化学习策略，使CPS具有自主学习和泛化能力，能够快速适应设计阶段未假设的未知情况。这些结果有望改变智能体在高维和异构环境中的交互方式，因此可能为探索前沿人工智能技术中的创造力提供深入的研究结果。 该项目的目标是推进强化学习的基础知识和科学方法，以实现CPS的泛化和可扩展性。受神经生物学和心理学最新研究的启发，该项目将为CPS设计一种新的技能驱动的智能控制方法，该方法可以学习更具表达力的扩展技能，以自主和自适应地处理未知情况，而无需进一步的人为干预。所提出的方法还将开发合作学习策略，以共享扩展的技能，以促进探索，并防止代理人混淆的行动细节。此外，该项目将发展自我激励的学习结构，以促进从分布式角度实现整个团队成功的全球目标。开发的方法和相关的架构将提供重要的见解和指导方针，以促进自主学习和概括CPS。研究和教育计划的整合将为CPS，人工智能，学习和控制领域的未来劳动力做好准备。通过各种学习方法，外展活动将建立CPS研究与少数群体（妇女和西班牙裔学生）、K-12和大学生之间的联系。该项目是对NSF CAREER 20-525征集的回应。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates new reinforcement learning (RL) approaches for cyber-physical autonomy to bridge the gap between current intelligent systems and human-level intelligence. The nature of many cyber-physical systems (CPS) is distributed, heterogeneous, and high-dimensional, making the hand-coded functions and task-specific information hard to design in the learning scheme. Large amount of training data is often required for achieving the desired performance, however this limits the generalization to other tasks. Hence, this project is to explore the new RL strategies to enable CPS with the capabilities of autonomous learning and generalization to rapidly adapt in unknown situations that were not assumed in the design phase. The results are expected to transform how agents interact in high-dimensional and heterogeneous environment, and therefore could potentially provide in-depth findings for exploring creativity in frontier Artificial Intelligence techniques. The goal of this project is to advance foundational knowledge and scientific methodologies of reinforcement learning for generalization and scalability in CPS. Motivated by the recent research in neurobiology and psychology, this project will design a new skill-driven intelligent control approach for CPS that can learn more expressive extended skills to autonomously and adaptively handle unknown situations without further human intervention. The proposed approach will also develop cooperative learning strategies to share with extended skills to facilitate exploration and prevent agents from getting confused by the action details. In addition, this project will develop self-motivated learning structures to contribute towards the global objectives for team-wide success in a distributed perspective. The developed methods and associated architectures will provide critical insights and guidelines to foster autonomous learning and generalization in CPS. The integration of research and education plans will prepare the future workforce in the fields of CPS, artificial intelligence, learning and control. The outreach activities will build connections between the CPS research, and minority groups (women and Hispanic students), K-12, and college students through various learning approaches.This project is in response to the NSF CAREER 20-525 solicitation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.