CAREER: Robust Online Decision Procedures for Societal Scale CPS

该研究项目将研究为公共交通，应急响应系统和电网等社会规模的网络物理系统设计顺序，非近视，在线决策程序的新方法，形成我们社区的关键基础设施。这些系统的在线优化需要采取行动，考虑紧密集成的空间，时间和人的维度，同时考虑由于系统和环境的变化而引起的不确定性。例如，应急响应管理系统（ERM）运营商必须最佳地调度救护车，并帮助卡车应对事故，同时考虑交通模式的变化和道路封闭。同样，运营电动汽车的公共交通机构必须考虑预期的出行需求来管理和调度车辆，同时考虑到整体电网负载来决定充电时间表。该项目提出的方法侧重于设计一个模块化和可重复使用的在线决策管道，该管道结合了在线规划方法（如蒙特-卡罗树搜索）与离线策略学习方法（如强化学习）的优势，有望提供更快的收敛速度和对环境变化的鲁棒性。建议的项目的研究活动的补充教育活动，专注于设计基于云的教学环境，可以帮助学生和运营商的先决条件域和统计知识的设计，管理和实验的决策程序。社会规模的CPS，我们研究的时空属性。空间方面指的是特定于位置的状态变量，如交通拥堵，交通需求，以及事件发生的频率在一个位置。时间方面指的是这些系统的动态性质-交通拥堵随着时间的推移而演变。非短视决策需要在不确定性下选择长期行动，同时考虑未来的影响和资源需求。该项目中提出的研究和教育工作的重点是回答这些系统的以下关键问题-首先，我们如何解决在高维空间中对未来状态/环境行为进行采样的挑战，同时也解决非平稳性的挑战？其次，我们如何解决对稳健、快速、非短视规划的需求，同时解决潜在的非平稳性？第三，我们如何让非计算机科学专业的学生和社区合作伙伴参与到使用项目中开创的方法构建的解决方案中来？所提出的方法涉及研究新的机器学习方法，如规范化流程设计生成模型和创新的方法来设计规划算法，使用策略增强的混合蒙特-卡罗树搜索方法。该项目的一个重要工作将集中在补充基础研究，设计一个基于云的可视化领域特定建模环境，可以帮助解释设计，操作和内省的方法，通过使用基于块的组合方法。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will study novel methods for designing sequential, non-myopic, online decision procedures for societal-scale cyber-physical systems such as public transit, emergency response systems, and power grid, forming the critical infrastructure of our communities. Online Optimization of these systems entails taking actions that consider the tightly integrated spatial, temporal, and human dimensions while accounting for uncertainty caused due to changes in the system and the environment. For example, emergency response management systems (ERM) operators must optimally dispatch ambulances and help trucks to respond to incidents while accounting for traffic pattern changes and road closures. Similarly, public transportation agencies operating electric vehicles must manage and schedule the vehicles considering the expected travel demand while deciding on charging schedules considering the overall grid load. The project's proposed approach focuses on designing a modular and reusable online decision-making pipeline that combines the advantages of online planning methods, such as Monte-Carlo Tree Search, with offline policy learning methods, such as reinforcement learning, promising to provide faster convergence and robustness to changes in the environment. The research activities of the proposed project are complemented by educational activities focusing on designing cloud-based teaching environments that can help students and operators with prerequisite domain and statistical knowledge to design, manage, and experiment with decision procedures.The societal-scale CPS that we study have spatial-temporal properties. The spatial aspect refers to the location-specific state variables such as traffic congestion, transportation demand, and the frequency with which incidents occur at a location. The temporal aspect refers to the dynamic nature of these systems---traffic congestion evolves over time. Non-myopic decisions entail selecting actions over time under uncertainty while accounting for future impact and demand for resources. The combined research and education efforts proposed in the project focus on answering the following critical questions for these systems - first, how do we solve the challenge of sampling future state/ environmental actions across a high-dimensional space while also tackling the challenge of non-stationarity? Second, how do we address the need for robust, fast non-myopic planning that also tackles potential non-stationarity? And third, how do we make it possible to engage non-computer science students and community partners with the solutions built using approaches pioneered in the project? The proposed approach involves investigating novel machine learning methods, such as normalizing flows for designing generative models and an innovative approach to design planning algorithms using a policy-augmented hybrid Monte-Carlo Tree Search approach. A significant effort of the project will focus on complementing fundamental research with the design of a cloud-based visual domain-specific modeling environment that can help explain the design, operation, and introspection of methods by using a block-based compositional approach. The work will be augmented with course modules and online tutorials accompanying the cloud-based environment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.