CAREER: A Skill-Driven Cooperative Learning Framework for Cyber-Physical Autonomy

该项目研究了用于网络物理自治的新的强化学习（RL）方法，以弥合当前智能系统与人类智能之间的差距。许多网络物理系统（CPS）具有分布式、异构和高维的特点，使得手工编码的功能和任务特定信息难以在学习方案中进行设计。为了达到预期的性能，通常需要大量的训练数据，然而这限制了对其他任务的推广。因此，该项目旨在探索新的强化学习策略，使CPS具有自主学习和泛化的能力，能够在设计阶段未假设的未知情况下快速适应。研究结果有望改变智能体在高维和异构环境中的相互作用方式，因此可能为探索前沿人工智能技术的创造力提供深入的发现。该项目的目标是推进强化学习的基础知识和科学方法，以实现CPS的泛化和可扩展性。受最近神经生物学和心理学研究的启发，该项目将为CPS设计一种新的技能驱动的智能控制方法，该方法可以学习更多具有表现力的扩展技能，以自主和自适应地处理未知情况，而无需进一步的人为干预。所提出的方法还将开发合作学习策略，与扩展技能共享，以促进探索并防止代理被动作细节所迷惑。此外，该项目将开发自我激励的学习结构，以实现分布式视角下团队范围内成功的全球目标。所开发的方法和相关架构将提供关键的见解和指导方针，以促进CPS的自主学习和泛化。研究和教育计划的整合将为CPS、人工智能、学习和控制领域的未来劳动力做好准备。外展活动将通过各种学习方法建立CPS研究与少数群体（妇女和西班牙裔学生），K-12和大学生之间的联系。本项目响应NSF CAREER 20-525招标。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project investigates new reinforcement learning (RL) approaches for cyber-physical autonomy to bridge the gap between current intelligent systems and human-level intelligence. The nature of many cyber-physical systems (CPS) is distributed, heterogeneous, and high-dimensional, making the hand-coded functions and task-specific information hard to design in the learning scheme. Large amount of training data is often required for achieving the desired performance, however this limits the generalization to other tasks. Hence, this project is to explore the new RL strategies to enable CPS with the capabilities of autonomous learning and generalization to rapidly adapt in unknown situations that were not assumed in the design phase. The results are expected to transform how agents interact in high-dimensional and heterogeneous environment, and therefore could potentially provide in-depth findings for exploring creativity in frontier Artificial Intelligence techniques. The goal of this project is to advance foundational knowledge and scientific methodologies of reinforcement learning for generalization and scalability in CPS. Motivated by the recent research in neurobiology and psychology, this project will design a new skill-driven intelligent control approach for CPS that can learn more expressive extended skills to autonomously and adaptively handle unknown situations without further human intervention. The proposed approach will also develop cooperative learning strategies to share with extended skills to facilitate exploration and prevent agents from getting confused by the action details. In addition, this project will develop self-motivated learning structures to contribute towards the global objectives for team-wide success in a distributed perspective. The developed methods and associated architectures will provide critical insights and guidelines to foster autonomous learning and generalization in CPS. The integration of research and education plans will prepare the future workforce in the fields of CPS, artificial intelligence, learning and control. The outreach activities will build connections between the CPS research, and minority groups (women and Hispanic students), K-12, and college students through various learning approaches.This project is in response to the NSF CAREER 20-525 solicitation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.