CAREER: Robust Online Decision Procedures for Societal Scale CPS

职业：社会规模 CPS 的稳健在线决策程序

• 批准号: 2238815
• 负责人: Abhishek Dubey
• 金额: $ 49.93万
• 依托单位: Vanderbilt University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-02-01 至 2028-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238815&HistoricalAwards=false
• 关键词: CAREER; Robust; Online; Decision; Procedures

This research project will study novel methods for designing sequential, non-myopic, online decision procedures for societal-scale cyber-physical systems such as public transit, emergency response systems, and power grid, forming the critical infrastructure of our communities. Online Optimization of these systems entails taking actions that consider the tightly integrated spatial, temporal, and human dimensions while accounting for uncertainty caused due to changes in the system and the environment. For example, emergency response management systems (ERM) operators must optimally dispatch ambulances and help trucks to respond to incidents while accounting for traffic pattern changes and road closures. Similarly, public transportation agencies operating electric vehicles must manage and schedule the vehicles considering the expected travel demand while deciding on charging schedules considering the overall grid load. The project's proposed approach focuses on designing a modular and reusable online decision-making pipeline that combines the advantages of online planning methods, such as Monte-Carlo Tree Search, with offline policy learning methods, such as reinforcement learning, promising to provide faster convergence and robustness to changes in the environment. The research activities of the proposed project are complemented by educational activities focusing on designing cloud-based teaching environments that can help students and operators with prerequisite domain and statistical knowledge to design, manage, and experiment with decision procedures.The societal-scale CPS that we study have spatial-temporal properties. The spatial aspect refers to the location-specific state variables such as traffic congestion, transportation demand, and the frequency with which incidents occur at a location. The temporal aspect refers to the dynamic nature of these systems---traffic congestion evolves over time. Non-myopic decisions entail selecting actions over time under uncertainty while accounting for future impact and demand for resources. The combined research and education efforts proposed in the project focus on answering the following critical questions for these systems - first, how do we solve the challenge of sampling future state/ environmental actions across a high-dimensional space while also tackling the challenge of non-stationarity? Second, how do we address the need for robust, fast non-myopic planning that also tackles potential non-stationarity? And third, how do we make it possible to engage non-computer science students and community partners with the solutions built using approaches pioneered in the project? The proposed approach involves investigating novel machine learning methods, such as normalizing flows for designing generative models and an innovative approach to design planning algorithms using a policy-augmented hybrid Monte-Carlo Tree Search approach. A significant effort of the project will focus on complementing fundamental research with the design of a cloud-based visual domain-specific modeling environment that can help explain the design, operation, and introspection of methods by using a block-based compositional approach. The work will be augmented with course modules and online tutorials accompanying the cloud-based environment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目将研究为社会规模的网络物理系统（例如公共交通、应急响应系统和电网）设计连续的、非短视的在线决策程序的新方法，形成我们社区的关键基础设施。这些系统的在线优化需要采取考虑紧密集成的空间、时间和人类维度的行动，同时考虑由于系统和环境的变化而引起的不确定性。例如，应急响应管理系统 (ERM) 操作员必须以最佳方式调度救护车并帮助卡车响应事故，同时考虑交通模式变化和道路封闭。同样，运营电动汽车的公共交通机构必须考虑预期的出行需求来管理和调度车辆，同时考虑整体电网负载来决定充电时间表。该项目提出的方法侧重于设计一个模块化且可重用的在线决策管道，该管道结合了蒙特卡罗树搜索等在线规划方法的优点与强化学习等离线策略学习方法，有望提供更快的收敛性和对环境变化的鲁棒性。拟议项目的研究活动辅以教育活动，重点是设计基于云的教学环境，可以帮助具有必备领域和统计知识的学生和操作员设计、管理和实验决策程序。我们研究的社会规模 CPS 具有时空属性。空间方面是指特定位置的状态变量，例如交通拥堵、交通需求以及某个位置发生事件的频率。时间方面是指这些系统的动态性质——交通拥堵随着时间的推移而演变。非短视决策需要在不确定的情况下随着时间的推移选择行动，同时考虑未来的影响和资源需求。该项目提出的联合研究和教育工作重点是回答这些系统的以下关键问题 - 首先，我们如何解决在高维空间中对未来状态/环境行为进行采样的挑战，同时应对非平稳性的挑战？其次，我们如何满足稳健、快速、非短视规划的需求，同时解决潜在的非平稳性问题？第三，我们如何才能让非计算机科学专业的学生和社区合作伙伴参与使用该项目首创的方法构建的解决方案？所提出的方法涉及研究新颖的机器学习方法，例如用于设计生成模型的标准化流程以及使用策略增强混合蒙特卡罗树搜索方法来设计规划算法的创新方法。该项目的一项重大工作将侧重于通过设计基于云的视觉特定领域建模环境来补充基础研究，该环境可以通过使用基于块的组合方法来帮助解释方法的设计、操作和内省。这项工作将通过伴随基于云的环境的课程模块和在线教程得到增强。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

HPRoP: Hierarchical Privacy-preserving Route Planning for Smart Cities

• DOI: 10.1145/3616874
• 发表时间: 2023-08
• 期刊: ACM Transactions on Cyber-Physical Systems
• 影响因子: 2.3
• 作者: F. Tiausas;K. Yasumoto;J. P. Talusan;H. Yamana;Hirozumi Yamaguchi;Shameek Bhattacharjee;Abhishek Dubey;Sajal K. Das

Act as You Learn: Adaptive Decision-Making in Non-Stationary Markov Decision Processes

• DOI: 10.48550/arxiv.2401.01841
• 发表时间: 2024-01
• 作者: Baiting Luo;Yunuo Zhang;Abhishek Dubey;Ayan Mukhopadhyay

Traffic Anomaly Detection Via Conditional Normalizing Flow

• DOI: 10.1109/itsc55140.2022.9922061
• 发表时间: 2022-10
• 期刊: 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)
• 作者: Zhuangwei Kang;Ayan Mukhopadhyay;A. Gokhale;Shijie Wen;Abhishek Dubey

Synchrophasor Data Event Detection using Unsupervised Wavelet Convolutional Autoencoders

• DOI: 10.1109/smartcomp58114.2023.00080
• 发表时间: 2023-06
• 期刊: 2023 IEEE International Conference on Smart Computing (SMARTCOMP)
• 作者: Jacob Buckelew;S. Basumallik;Vasavi Sivaramakrishnan;Ayan Mukhopadhyay;Amal Srivastava;Abhishek Dubey

Decision Making in Non-Stationary Environments with Policy-Augmented Search

通过策略增强搜索在非平稳环境中做出决策

• 发表时间: 2024
• 期刊: International Conference on Autonomous Agents and Multiagent Systems
• 作者: Pettet, Ava;Zhang, Yunuo;Luo, Baiting;Wray, Kyle;Baier, Hendrik;Laszka, Aron;Dubey, Abhishek;Mukhopadhyay, Ayan
• 通讯作者: Mukhopadhyay, Ayan