Collaborative Research: Risk-Averse Control of Markov Systems with Model Uncertainty

该项目重点研究随时间随机演化且观察者无法准确了解其基本特征的系统中决策的数学理论和计算方法。该研究将以连贯的方式解决如何对此类系统中的风险进行建模以及如何在风险规避范式内控制它们。这将通过开发动态风险评估程序（称为风险过滤器）并采用自适应鲁棒控制技术来实现。该项目的成果将直接推动和推动科学与工程的进步，在医学、工程、经济、金融、库存管理和保险等应用领域具有潜在的应用前景。将特别关注推广拟议的研究及其在这些应用领域的影响。特别是，这将通过为研究生和本科生（包括来自代表性不足群体的学生）提供建议、在流行的、国际和当地论坛上的演讲以及通过科学期刊和书籍出版物传播结果来实现。马尔可夫决策过程的经典理论和实践已被证明提供了一个强大而成功的工具包，可以在决策者能够获得充分已知（准确）的潜在马尔可夫动力系统模型的情况下生成最佳或次优决策策略，并采取行动以优化预期累积成本或因决策者的行为而产生的奖励。然而，一方面，在许多决策过程中，决策者需要考虑决策的累积奖励和累积风险之间的权衡。本研究项目所依据的风险规避决策标准和风险过滤器理论非常适合此类目的。另一方面，决策过程中的典型情况是不确切地知道潜在的马尔可夫动力系统的模型。通常，这种模型是底层马尔可夫系统的半充分形式化，即系统的结构动力学特征被充分建模，但缺少相关模型参数的精确知识。在这种情况下，我们说决策者面临模型不确定性。拟议研究的一部分将致力于开发通过自适应鲁棒随机控制框架解决这一问题的方法。因此，所提出的研究以连贯且新颖的方式解决了马尔可夫系统决策的两个重要方面：风险规避决策标准和模型不确定性。风险过滤器理论将与自适应鲁棒控制方法相结合，从而产生新颖的动态规划方程，并为此建立新的数值方法。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project focuses on mathematical theory and computational methods of decision-making in systems that evolve randomly in time and whose essential characteristics are not precisely known to the observer. The research will address in a coherent way how to model risk in such systems and how to control them within the risk-averse paradigm. This will be accomplished by developing dynamic risk-assessment procedures, called risk filters, and by employing adaptive robust control techniques. The outcome of the project will directly advance and promote the progress of science and engineering, with potential applications in applied areas such as medical sciences, engineering, economics, finance, inventory management and insurance. Special attention will be given to popularizing the proposed research and its impact in these applied fields. In particular, this will be achieved through advising of graduate and undergraduate students, including students from underrepresented groups, presentations at popular, international and local forums, and dissemination of the results via scientific journal and book publications.The classical theory and practice of Markov decision processes have proven to provide a powerful and successful toolkit for generating optimal or sub-optimal decision strategies in situations where the decision maker has access to adequately known (accurate) model of the underlying Markovian dynamical system, and acts so to optimize the expected cumulative cost or reward arising from the decision maker's actions. However, on the one hand, in many decision-making processes the decision maker needs to account for the trade-off between the cumulative award and cumulative risk of the decision. Risk-averse decision criteria underlying this research project and the theory of risk filters are ideally suited for such purposes. On the other hand, it is a typical situation in decision making processes that the model of the underlying Markovian dynamical system is not known exactly. Frequently, such model is a semi-adequate formalization of the underlying Markovian system, in the sense that the structural dynamical features of the system are modeled adequately, but precise knowledge of relevant model parameters is missing. In such cases, we say that the decision maker faces model uncertainty. Part of the proposed research will be devoted to develop methodologies that address this issue through adaptive robust stochastic control framework. Thus, the proposed research addresses in a coherent and novel way two important aspects of decision making in Markov systems: risk-averse decision criteria and model uncertainty. The theory of risk filters will be combined with the adaptive robust control methodology that will lead to novel dynamic programming equations, for which new numerical methods will be established.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.