CAREER: Towards an Intermittent Learning Framework for Smart and Efficient Cyber-Physical Autonomy

该项目扩展了如何将强化学习框架用于网络物理系统（CPS）的自治。该研究利用了间歇性强化，即不是每次完成期望的反应都给予奖励。这与传统的强化学习机制不同，在传统的强化学习机制中，在线训练中的每一点都有奖励。这个框架的新颖之处在于，它可以展示如何在罕见事件或嘈杂和对抗性数据影响这些算法的训练和性能时使用强化学习。这项工作将通过与瑞典和英国的国际伙伴关系，在协同公路货运和协作机器人试验台上进行验证。该项目包括将高中生融入机器学习领域的挑战性问题的活动，其动机是无人机竞赛。本研究的目的是通过加深学习、控制、博弈论和CPS社区之间的联系来扩展基础知识。该方法是：(i)统一关于弹性、带宽效率、鲁棒性和其他方面的工程学习的新观点，这些是用最先进的方法无法实现的；（ii）为CPS开发间歇性深度学习方法，以减轻传感器攻击，并处理传感能力有限的情况；（iii）将CPS中的非均衡博弈论学习与决策、理性和信息使用根本不同的组成部分结合起来；(4)研究将学习转移到新平台的方法。该项目的教育和外联部分包括将导致技术转让的实习、特别侧重于接触代表性不足的少数民族和妇女的夏令营，以及通过学生交流项目与瑞典和英国的机构合作。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project expands how reinforcement learning frameworks can be used for Cyber-Physical Systems (CPS) for autonomy. The research utilizes intermittent reinforcement, where a reward is not given every time the desired response is performed. This differs from traditional reinforcement learning mechanisms, in which a reward is given for each point during online training. What is novel in this framework is that it can demonstrate how reinforcement learning can be used when rare events, or noisy and adversarial data, can affect the training and performance of these algorithms. The work will be validated on collaborative road freight transport and collaborative robotics testbeds, through international partnerships with Sweden and the United Kingdom. The project includes activities that integrate high-school students into challenging problems in machine learning areas, motivated through drone racing competitions.The goal of this research is to expand foundational knowledge through deepened ties between the learning, control, game theory, and CPS communities. The approach is to, (i) unify new perspectives of learning in engineering with respect to resiliency, bandwidth efficiency, robustness, and other aspects that cannot be achieved with the state-of-the-art approaches; (ii) develop intermittent deep learning methods for CPS that can mitigate sensor attacks and can handle cases of limited sensing capabilities; (iii) incorporate nonequilibrium game-theoretic learning in CPS with components whose decision-making, rationality, and information usage are fundamentally different; and (iv) investigate ways to transfer learning to new platforms. The project's education and outreach component includes internships that will lead to technology transfer, summer camps with a special focus on reaching out to underrepresented minorities and women, and collaboration with institutions in Sweden and the United Kingdom through student exchange programs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.