Markov decision models for dynamic risk management

本文主要研究在动态风险管理背景下求解非线性多期马尔可夫决策模型的新方法。它涉及到一个设计过程，高度复杂的动态模型适用于风险管理活动。这个过程自然可以分为三个步骤：首先，根据系统的结构和控制目标及约束条件，建立状态变量的理论模型。该模型以一组状态指标作为主要的风险源，用马尔可夫过程描述状态动态。其次，获取最新的数据库，用于模型参数的估计和模型本身的验证。在估计过程中应用了高效的统计和模拟算法，如改进的期望和最大化算法。第三，将制定和求解马尔可夫决策模型，以获得最佳决策规则及其实际意义。以风险价值（Value at Risk）和条件风险价值（Conditional Value at Risk）作为风险最小化的目标函数，通过大规模随机动态规划技术得到最优策略。本研究的一个新奇是多周期非线性随机规划模型的分解求解技术，研究的目的是建立不确定性动态控制的现实模型，并研究理论上难以解决的问题的有效求解算法。

The proposed research mainly focuses on a new method for solving nonlinear multiperiod Markov decision models in a setting of dynamic risk management. It involves a design process for highly sophisticated dynamic models applicable to risk management activities. This process can be naturally divided into three steps.First, theoretical models for state variables will be established based on the structure of the system and the control objectives and constraints. The proposed model takes  a set of state indicators as primary sources of risk and characterizes the state dynamics with Markov processes.Second, up-to-date databases will be acquired and used for the estimation of model parameters and validation of the model itself. Highly efficient statistical and simulation algorithms, such as the adapted Expectation and Maximization algorithm, are applied in the estimation procedures.Third, Markov decision models will be formulated and solved to obtain the optimal decision rules and their practical implications. Using a popular risk management measure, such as Value at Risk and Conditional Value at Risk, as the objective function of minimizing risk, the optimal policy is obtained through large-scale stochastic dynamic programming techniques. A novelty of this research is the  decomposition solution technique for multiperiod nonlinear stochastic programming models.The intention of the research program is to build  realistic models for dynamic control under uncertainty and to study efficient solution algorithms for theoretically intractable problems.