Collaborative Research: Risk-Averse Control of Markov Systems with Model Uncertainty

该项目的重点是在随机演变的系统中进行决策的数学理论和计算方法，其基本特征对观察者来说并不精确。该研究将以一种连贯的方式解决如何在这种系统中建模风险以及如何在风险规避范式中控制它们。这将通过制定动态风险评估程序（称为风险过滤器）和采用自适应鲁棒控制技术来实现。该项目的成果将直接推动和促进科学和工程的进步，在医学、工程、经济、金融、库存管理和保险等应用领域具有潜在的应用价值。将特别注意推广拟议的研究及其在这些应用领域的影响。特别是，这将通过指导研究生和本科生，包括来自代表性不足群体的学生，在流行的，国际和地方论坛上的演讲，马尔可夫决策过程的经典理论和实践已被证明是一个强大而成功的工具包，用于生成最优或次优决策模型。最优决策策略的情况下，决策者有机会充分了解（准确）模型的基础马尔可夫动态系统，并采取行动，以优化预期的累积成本或奖励所产生的决策者的行动。然而，一方面，在许多决策过程中，决策者需要考虑决策的累积奖励和累积风险之间的权衡。本研究项目的风险规避决策标准和风险过滤理论非常适合于这种目的。另一方面，它是一个典型的情况下，在决策过程中，潜在的马尔可夫动力系统的模型是不知道确切的。通常情况下，这种模型是一个半充分的形式化的基本马尔可夫系统，在这个意义上说，系统的结构动力学特性建模充分，但相关的模型参数的精确知识是失踪。在这种情况下，我们说决策者面临模型的不确定性。部分拟议的研究将致力于开发方法，通过自适应鲁棒随机控制框架来解决这个问题。因此，建议的研究地址在一个连贯的和新颖的方式在马尔可夫系统中的决策的两个重要方面：风险厌恶的决策标准和模型的不确定性。风险过滤器的理论将与自适应鲁棒控制方法相结合，这将导致新的动态规划方程，新的数值方法将被建立。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

This project focuses on mathematical theory and computational methods of decision-making in systems that evolve randomly in time and whose essential characteristics are not precisely known to the observer. The research will address in a coherent way how to model risk in such systems and how to control them within the risk-averse paradigm. This will be accomplished by developing dynamic risk-assessment procedures, called risk filters, and by employing adaptive robust control techniques. The outcome of the project will directly advance and promote the progress of science and engineering, with potential applications in applied areas such as medical sciences, engineering, economics, finance, inventory management and insurance. Special attention will be given to popularizing the proposed research and its impact in these applied fields. In particular, this will be achieved through advising of graduate and undergraduate students, including students from underrepresented groups, presentations at popular, international and local forums, and dissemination of the results via scientific journal and book publications.The classical theory and practice of Markov decision processes have proven to provide a powerful and successful toolkit for generating optimal or sub-optimal decision strategies in situations where the decision maker has access to adequately known (accurate) model of the underlying Markovian dynamical system, and acts so to optimize the expected cumulative cost or reward arising from the decision maker's actions. However, on the one hand, in many decision-making processes the decision maker needs to account for the trade-off between the cumulative award and cumulative risk of the decision. Risk-averse decision criteria underlying this research project and the theory of risk filters are ideally suited for such purposes. On the other hand, it is a typical situation in decision making processes that the model of the underlying Markovian dynamical system is not known exactly. Frequently, such model is a semi-adequate formalization of the underlying Markovian system, in the sense that the structural dynamical features of the system are modeled adequately, but precise knowledge of relevant model parameters is missing. In such cases, we say that the decision maker faces model uncertainty. Part of the proposed research will be devoted to develop methodologies that address this issue through adaptive robust stochastic control framework. Thus, the proposed research addresses in a coherent and novel way two important aspects of decision making in Markov systems: risk-averse decision criteria and model uncertainty. The theory of risk filters will be combined with the adaptive robust control methodology that will lead to novel dynamic programming equations, for which new numerical methods will be established.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.