权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Optimality and Robustness in Piecewise-Deterministic Systems

分段确定性系统的最优性和鲁棒性

基本信息

批准号：
2111522
负责人：
Alexander Vladimirsky
金额：
$ 46.68万
依托单位：
Cornell University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-09-01 至 2024-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2111522&HistoricalAwards=false
关键词：
Optimality Robustness Piecewise Deterministic Systems

项目摘要

This project focuses on quantifying and actively managing uncertainties resulting from random switches in global environments. An abrupt ecological change, a new disruptive technology, a global economic downturn, or an emerging pandemic – any such game-changer event may transform the planning environment and shift the priorities of decision makers. Reactive planning is often the norm in practice, with the working assumption that global mode switches are too rare and unlikely to take them into account. But if a statistical characterization of such switches is available, using it in strategic planning can significantly improve the performance of the controlled system. Models with these features arise naturally in many research areas including economics, behavioral ecology, public policy, robotic navigation, evolutionary biology, and security applications (e.g., preventing illegal logging or wildlife poaching). PI will develop efficient numerical methods for controlling such systems, focusing on trade-offs between the average-case optimality and robustness (interpreted as minimizing the probability of undesirable outcomes.) This project will support 2 graduate students in each of the first two years and 1 graduate student in the third year. Piecewise-Deterministic Markov Processes (PDMPs) provide an excellent framework for modeling large-scale stochastic perturbations of the global environment. The aleatoric uncertainty due to such perturbations is an important feature of realistic control problems, but until recently it has attracted far less attention in mathematical literature than the diffusive perturbations studied in ``classical'' stochastic optimal control theory. Practitioners often want to model these environment-switch uncertainties as well as time-structured information accumulation patterns present in their applications. Moreover, it may not be enough for them to optimize the expected value of the outcome. They often need to maximize the probability of good outcomes while imposing constraints on the worst-case scenario. To accomplish this, we need to modify the partial differential equations (PDEs) encoding the optimal behavior, and this presents a range of new computational challenges: free boundaries, discontinuities, higher dimensionality of the state space, and larger systems of coupled nonlinear PDEs. We propose to study the trade-offs involved in using such modified models and to develop numerical methods to solve them efficiently. In the PDMP setting, even the traditional risk-neutral approach of optimizing the expected performance can be computationally costly since it involves solving a coupled system of Hamilton-Jacobi-Bellman (HJB) equations. We develop several approaches for decreasing this computational cost by constructing new discretization schemes and leveraging efficient methods previously developed for fully deterministic problems. We also extend our recent approach for optimizing the Cumulative Distribution Function (CDF) of the total cost incurred by a stochastic switching system. This is accomplished by solving a different system of HJB equations on an expanded state space, with ``threshold-optimal'' controls recovered for all starting configurations and all threshold values simultaneously. We further investigate the trade-offs between conflicting optimization criteria and several notions of robustness.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的重点是量化和积极管理全球环境中随机变化造成的不确定性。突然的生态变化、新的颠覆性技术、全球经济衰退或新出现的流行病-任何此类改变游戏规则的事件都可能改变规划环境并改变决策者的优先事项。反应式规划通常是实践中的规范，其工作假设是全局模式切换太罕见，不太可能考虑它们。但是，如果这种开关的统计特性是可用的，在战略规划中使用它可以显着提高受控系统的性能。具有这些特征的模型自然出现在许多研究领域，包括经济学、行为生态学、公共政策、机器人导航、进化生物学和安全应用（例如，防止非法砍伐或偷猎野生动物）。PI将开发用于控制此类系统的有效数值方法，重点是平均情况下的最优性和鲁棒性之间的权衡（解释为最小化不良结果的概率）。该项目将在前两年每年支持2名研究生，第三年支持1名研究生。分段确定性马尔可夫过程（PDMP）为全球环境的大规模随机扰动建模提供了一个很好的框架。任意的不确定性，由于这种扰动是现实控制问题的一个重要特征，但直到最近，它已经吸引了远远低于数学文献中的注意力比扩散扰动研究的“经典”随机最优控制理论。从业者经常希望对这些环境切换的不确定性以及应用程序中存在的时间结构信息积累模式进行建模。此外，这可能不足以让他们优化结果的预期价值。他们往往需要最大限度地提高好结果的可能性，同时对最坏的情况施加限制。为了实现这一点，我们需要修改编码最优行为的偏微分方程（PDE），这带来了一系列新的计算挑战：自由边界，不连续性，状态空间的高维性，以及耦合非线性PDE的更大系统。我们建议研究使用这种修改后的模型所涉及的权衡，并开发数值方法来有效地解决这些问题。在Pestrian设置中，即使是优化预期性能的传统风险中性方法也可能在计算上代价高昂，因为它涉及求解Hamilton-Jacobi-Bellman（HJB）方程的耦合系统。我们开发了几种方法，通过构建新的离散化方案和利用以前开发的完全确定性问题的有效方法来降低计算成本。我们还扩展了我们最近的方法，优化的累积分布函数（CDF）的随机切换系统所产生的总成本。这是通过在扩展的状态空间上求解不同的HJB方程组来实现的，同时对所有起始配置和所有阈值恢复“阈值最优”控制。我们进一步研究了相互冲突的优化标准和几个鲁棒性概念之间的权衡。该奖项反映了NSF的法定使命，并被认为是值得支持的，通过使用基金会的智力价值和更广泛的影响审查标准进行评估。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Optimal Path-Planning With Random Breakdowns

随机故障的最优路径规划

DOI：
10.1109/lcsys.2021.3130193
发表时间：
2022
期刊：
IEEE Control Systems Letters
影响因子：
3
作者：
Gee, Marissa;Vladimirsky, Alexander
通讯作者：
Vladimirsky, Alexander

Surveillance Evasion Through Bayesian Reinforcement Learning

通过贝叶斯强化学习规避监视

DOI：
发表时间：
2023
期刊：
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics
影响因子：
0
作者：
Qi, Dongping;Bindel, David;Vladimirsky, Alexander
通讯作者：
Vladimirsky, Alexander

Stochastic Optimal Control of a Sailboat

帆船的随机最优控制