CPS: Medium: Collaborative Research: Developing Data-driven Robustness and Safety from Single Agent Settings to Stochastic Dynamic Teams: Theory and Applications

这一网络物理系统(CPS)项目将在方法学上取得基础性进展，以实现由智能交通信号控制系统中的问题驱动的基于强化学习(RL)的安全和稳健的控制算法解决方案。计算、通信、存储和传感方面的最新进展导致了现代网络物理系统(CPSS)(如智能交通系统)对基于数据驱动的学习的决策和控制的需求。在这样的系统中，决策代理需要以安全和健壮的方式操作，同时在具有需要遵守的约束的复杂环境中工作。该项目将以智能交通系统中的交通信号控制系统为激励CPS应用和评估平台，在稳健的RL解决方案和安全约束的RL方面取得基础性进展，并提供可证明的保证。这项工作还将侧重于促进课程开发，从代表性不足的群体中招收学生，本科生参与研究，K-12外联，以及通过讲习班、会议和研讨会进行研究社区外联。研究人员将与公司和其他利益攸关方沟通研究结果，并为他们提供方法论方面的教育材料。所述技术方法包括：1.结合模型类知识、未来预测和稳健性表征以及非策略方法的健壮RL解决方案，以解决由于使用模拟器/仿真器或离线数据而引起的分布移位和数据匮乏；以及2.使用无模型方法和函数近似方法的高效、安全和受限的RL算法，以及用于部分观测系统的方法。为了用激励性的CPS应用程序结束环路，RL算法将在交通信号控制的背景下通过使用两个仪表站的模型进行基于模拟的综合评估进行评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Cyber-Physical Systems (CPS) project will make foundational methodological advances that enable safe and robust reinforcement learning (RL)-based control algorithmic solutions that are driven by problems in smart traffic signal control systems. Recent advances in computation, communication, storage, and sensing have led to a demand for data-driven learning-based decision-making and control in modern cyber-physical systems (CPSs), such as smart transportation systems. In such systems, decision-making agents need to operate safely and in a robust manner while working in complex environments with constraints that need to be respected. This project will develop foundational advances in robust RL solutions, and safe and constrained RL with provable guarantees by taking traffic signal control systems within smart transportation systems as our motivating CPS application and evaluation platform. This work will additionally focus on advancing curriculum development, recruitment of students from under-represented groups, involvement of undergraduate students in research, K-12 outreach, and also research community outreach via workshops, conference sessions, and seminars. The researchers will interface with companies and other stakeholders to communicate the results of the research as well as provide them with educational material on methodology. The technical approaches include: 1. Robust RL solutions incorporating model class knowledge, use of future predictions and robustness characterizations, and off-policy methods to address distributional shifts and data paucity arising from the use of a simulator/emulator or offline data; and 2. Efficient, safe, and constrained RL algorithms using model-free approaches and function-approximated methods, and also methods for partially-observed systems. To close the loop with the motivating CPS application, the RL algorithms will be evaluated in the context of traffic signal control via a comprehensive simulation-based evaluation using models of two instrumented sites.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.