Collaborative Research: CPS: Small: An Integrated Reactive and Proactive Adversarial Learning for Cyber-Physical-Human Systems

协作研究：CPS：小型：网络-物理-人类系统的集成反应式和主动式对抗学习

• 批准号: 2227185
• 负责人: Kyriakos G Vamvoudakis
• 金额: $ 25万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2227185&HistoricalAwards=false
• 关键词: Collaborative; Research; CPS; Small; Integrated

The gradual deployment of self-driving cars will inevitably lead to the emergence of a new important class of cyber-physical-human systems where autonomous vehicles interact with human-driven vehicles via on-board sensors or vehicle-to-vehicle communications. Reinforcement learning along with control theory can help meet the safety requirements for real-time decision making and Level 5 autonomy in self-driving vehicles. However, it is widely known that conventional reinforcement learning policies are vulnerable to adversarial or non-adversarial perturbations to their observations, similar to adversarial examples for classifiers and/or reward (packet) drops of the learning. Such issues are exacerbated by concerns of addressing resiliency as the use of open communication and control platforms for autonomy becomes essential, and as the industry continues to invest in such systems. Decision making mechanisms, designed to incorporate agility with the help of reinforcement learning, allow self-adaptation, self-healing, and self-optimization. This research will contribute and unify the body of knowledge of several diverse fields including reinforcement learning, security, automatic control, and transportation for resilient autonomy with humans-in-the-loop.In this project, to counter action and observation manipulation as well as reward drops, the principal investigators will leverage proactive switching policies that aim (i) to provide robustness to adversarial inputs and reward drops in the closed-loop reinforcement learning mechanisms, (ii) to increase the cost of manipulation by deception, (iii) to limit the exposure of vulnerable actions and observations, and (iv) to provide stability, optimality, and robustness guarantees. Ultimately, the investigators will develop fundamental contributions to each of the above-mentioned fields and amalgamate these fields to provide a unique synthesis framework. The outcomes of this project will increase levels of confidence in autonomous technologies from ethical perspectives by providing an underpinning for curtailing accidents. The proposed framework can be extended to other key enablers of the global economy, including smart and connected cities, healthcare, and networked actions of smart systems while decreasing environmental pollution and minimizing the adverse environmental impacts on human health. The project will train the next generation of students from various levels, ages, and cultures through well-coordinated, level appropriate involvement in research and educational activities while providing a unique opportunity for the students to appreciate efficient, autonomous, and low-cost designs. This project will also contribute to future engineering curricula, pursue a substantial integration of research and education, and provide opportunities to engage students from the underrepresented group.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自动驾驶汽车的逐步部署将不可避免地导致一类新的重要的网络-物理-人类系统的出现，其中自动驾驶汽车通过车载传感器或车对车通信与人类驾驶的汽车进行交互。强化学习沿着控制理论可以帮助满足自动驾驶车辆实时决策和5级自主的安全要求。然而，众所周知，传统的强化学习策略容易受到对其观察结果的对抗性或非对抗性扰动的影响，类似于分类器和/或学习的奖励（数据包）丢弃的对抗性示例。随着使用开放式通信和控制平台实现自主变得至关重要，以及随着行业继续投资于此类系统，解决弹性问题的担忧加剧了这些问题。决策机制，旨在结合敏捷性与强化学习的帮助下，允许自适应，自我修复和自我优化。这项研究将有助于并统一几个不同领域的知识体系，包括强化学习，安全，自动控制和运输，以实现弹性自主与人在环。在这个项目中，为了对抗行动和观察操纵以及奖励下降，主要调查人员将利用积极主动的转换政策，目的是：在闭环强化学习机制中提供对抗性输入和奖励下降的鲁棒性，（ii）增加欺骗操纵的成本，（iii）限制易受攻击的动作和观察的暴露，以及（iv）提供稳定性，最优性，和鲁棒性保证。最终，研究人员将为上述每个领域做出基本贡献，并合并这些领域，以提供一个独特的综合框架。该项目的成果将通过为减少事故提供基础，从道德角度提高对自主技术的信心。拟议的框架可以扩展到全球经济的其他关键推动因素，包括智能和互联城市，医疗保健和智能系统的网络行动，同时减少环境污染并最大限度地减少对人类健康的不利环境影响。该项目将通过良好协调，适当参与研究和教育活动，培养来自不同层次，年龄和文化的下一代学生，同时为学生提供欣赏高效，自主和低成本设计的独特机会。该项目还将有助于未来的工程课程，追求研究和教育的实质性整合，并提供机会，使学生从代表性不足的group.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

Cooperative Finitely Excited Learning for Dynamical Games

• DOI: 10.1109/tcyb.2023.3274908
• 发表时间: 2023-05
• 期刊: IEEE Transactions on Cybernetics
• 影响因子: 11.8
• 作者: Yongliang Yang;H. Modares;K. Vamvoudakis;F. Lewis

Game Theory for Autonomy: From Min-Max Optimization to Equilibrium and Bounded Rationality Learning

• DOI: 10.23919/acc55779.2023.10156432
• 发表时间: 2023-05
• 期刊: 2023 American Control Conference (ACC)
• 作者: K. Vamvoudakis;Filippos Fotiadis;J. Hespanha;Raphael Chinchilla;Guosong Yang;Mushuang Liu;J. Shamma;Lacra Pavel

Decentralized Multi-Agent Motion Planning in Dynamic Environments

动态环境中的分散式多智能体运动规划

• DOI: 10.23919/acc55779.2023.10156024
• 发表时间: 2023
• 期刊: 2023 American Control Conference (ACC
• 作者: Netter, Josh;Vamvoudakis, Kyriakos G.
• 通讯作者: Vamvoudakis, Kyriakos G.

Verification of Adversarially Robust Reinforcement Learning Mechanisms in Aerospace Systems

航空航天系统中对抗性鲁棒强化学习机制的验证

• DOI: 10.2514/6.2023-1070
• 发表时间: 2023
• 期刊: Proc. AIAA SCITECH 2023 Forum
• 作者: Seo, Taehwan;Sahoo, Prachi P.;Vamvoudakis, Kyriakos G.
• 通讯作者: Vamvoudakis, Kyriakos G.

An Online Model-Following Projection Mechanism Using Reinforcement Learning

• DOI: 10.1109/tac.2023.3243165
• 发表时间: 2023-02
• 期刊: IEEE Transactions on Automatic Control
• 影响因子: 6.8
• 作者: M. Abouheaf;Hashim A. Hashim-Hashim-A.-Hashim-36452482;M. Mayyas;K. Vamvoudakis