FRR: Collaborative Research: Collaborative Learning for Multi-robot Systems with Model-enabled Privacy Protection and Safety Supervision

FRR：协作研究：具有模型支持的隐私保护和安全监督的多机器人系统协作学习

• 批准号: 2219487
• 负责人: Yongqiang Wang
• 金额: $ 38.76万
• 依托单位: Clemson University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2219487&HistoricalAwards=false
• 关键词: FRR; Collaborative; Research; Learning; Multi

This project provides a collaborative reinforcement learning approach for multi-robot systems that ensures safety and is privacy-preserving. The approach enables robots to continuously learn and adapt to dynamic situations within the systems constraints. Moreover, this new approach ensures that the private information of the participating robots, such as identity and position, can be protected in collaborative tasks involving multiple participating robots. While the approach for multi-robot learning is general, it has application to intelligent transportation systems on roadways driven by autonomous and semi-autonomous cars and trucks. A demonstration of this approach is to be conducted in a full-scale test environment of a realistic urban setting.This project combines model-based safety with model-free reinforcement learning to enable reinforcement learning's applicability to safety-critical collaborative multi-robot systems. It will first address single-robot reinforcement learning using a deep Koopman-based safety regulation for general nonlinear robotic systems to guarantee safety while retaining learning efficiency. The result will then be extended to multi-robot collective reinforcement learning where robots are deployed in shared, contested, or resource-constrained environments. By exploiting the inherent dynamics of collaborative learning, the project will also enable dynamics-based privacy protection for collected and shared data during learning. Different from conventional privacy mechanisms that either trade accuracy for privacy or incur heavy computation/communication overhead, the dynamics-enabled privacy approach can maintain learning optimality while incurring little computation/communication overhead. The algorithms and frameworks will be evaluated using both numerical simulations and experiments with real connected vehicles on real tracks. Results of the project will be used to enrich both graduate and undergraduate courses. The PIs will also use existing various on-going outreach opportunities to energize interests in STEM in K-12 students and community college technicians.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目为多机器人系统提供了一种协作强化学习方法，确保了安全性和隐私保护。该方法使机器人能够在系统约束范围内不断学习和适应动态情况。此外，这种新的方法确保了在涉及多个参与机器人的协作任务中，参与机器人的身份和位置等隐私信息能够得到保护。虽然多机器人学习的方法是通用的，但它适用于由自动和半自动汽车和卡车驾驶的道路上的智能交通系统。该方法将在真实城市环境的全尺寸测试环境中进行演示，该项目将基于模型的安全性与无模型强化学习相结合，以使强化学习能够适用于安全关键的协作多机器人系统。它将首先解决单机器人强化学习，使用针对一般非线性机器人系统的基于深度库普曼的安全规则，以在保证安全的同时保持学习效率。然后将结果扩展到多机器人集体强化学习，其中机器人被部署在共享、竞争或资源受限的环境中。通过利用协作学习的内在动力，该项目还将为学习期间收集和共享的数据提供基于动态的隐私保护。与传统的隐私机制不同，动态隐私机制要么以精度换取隐私，要么导致大量的计算/通信开销，动态使能隐私方法可以在引起很小的计算/通信开销的情况下保持学习的最优性。这些算法和框架将通过数值模拟和在真实轨道上的真实连接车辆的实验来进行评估。该项目的成果将用于丰富研究生和本科课程。PIS还将利用现有的各种正在进行的外展机会，激发K-12学生和社区大学技术员对STEM的兴趣。该项目由跨部门机器人基础研究计划支持，该计划由工程学指导委员会(ENG)和计算机和信息科学与工程指导委员会(CEISE)联合管理和资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Differentially-Private Distributed Optimization with Guaranteed Optimality

• DOI: 10.1109/cdc49753.2023.10383285
• 发表时间: 2023-12
• 期刊: 2023 62nd IEEE Conference on Decision and Control (CDC)
• 作者: Yongqiang Wang;A. Nedić

Distributed Optimization with Noisy Information Sharing

• DOI: 10.23919/ccc58697.2023.10240327
• 发表时间: 2023-07
• 期刊: 2023 42nd Chinese Control Conference (CCC)
• 作者: Yongqiang Wang

A Robust Dynamic Average Consensus Algorithm that Ensures both Differential Privacy and Accurate Convergence

• DOI: 10.1109/cdc49753.2023.10383541
• 发表时间: 2022-11
• 期刊: 2023 62nd IEEE Conference on Decision and Control (CDC)
• 作者: Yongqiang Wang

Differentially-private Distributed Algorithms for Aggregative Games with Guaranteed Convergence

保证收敛的聚合博弈的差分私有分布式算法

• DOI: 10.1109/tac.2024.3351068
• 发表时间: 2024
• 期刊: IEEE Transactions on Automatic Control
• 影响因子: 6.8
• 作者: Wang, Yongqiang;Nedić, Angelia
• 通讯作者: Nedić, Angelia

Ensuring both Almost Sure Convergence and Differential Privacy in Nash Equilibrium Seeking on Directed Graphs

确保有向图纳什均衡搜索中的几乎确定收敛和差分隐私

• DOI: 10.1109/tac.2024.3366037
• 发表时间: 2024
• 期刊: IEEE Transactions on Automatic Control
• 影响因子: 6.8
• 作者: Wang, Yongqiang;Başar, Tamer
• 通讯作者: Başar, Tamer