Markov decision models for dynamic risk management

本文主要研究一种求解动态风险管理环境下非线性多周期马尔可夫决策模型的新方法。它涉及适用于风险管理活动的高度复杂动态模型的设计过程。这个过程自然可以分为三个步骤。首先，根据系统结构、控制目标和约束条件，建立状态变量的理论模型。该模型将一组状态指标作为风险的主要来源，并用马尔可夫过程表征状态动态。其次，将获得最新的数据库，并将其用于模型参数的估计和模型本身的验证。高效的统计和仿真算法，如适应的期望和最大化算法，应用于估计过程。第三，建立并求解马尔可夫决策模型，得到最优决策规则及其实际意义。采用风险值和条件风险值作为风险最小化的目标函数，通过大规模随机动态规划技术得到最优策略。本研究的一个新颖之处是多周期非线性随机规划模型的分解求解技术。本研究计划旨在建立不确定情况下动态控制的现实模型，并研究理论上难以解决问题的有效求解算法。

The proposed research mainly focuses on a new method for solving nonlinear multiperiod Markov decision models in a setting of dynamic risk management. It involves a design process for highly sophisticated dynamic models applicable to risk management activities. This process can be naturally divided into three steps.First, theoretical models for state variables will be established based on the structure of the system and the control objectives and constraints. The proposed model takes  a set of state indicators as primary sources of risk and characterizes the state dynamics with Markov processes.Second, up-to-date databases will be acquired and used for the estimation of model parameters and validation of the model itself. Highly efficient statistical and simulation algorithms, such as the adapted Expectation and Maximization algorithm, are applied in the estimation procedures.Third, Markov decision models will be formulated and solved to obtain the optimal decision rules and their practical implications. Using a popular risk management measure, such as Value at Risk and Conditional Value at Risk, as the objective function of minimizing risk, the optimal policy is obtained through large-scale stochastic dynamic programming techniques. A novelty of this research is the  decomposition solution technique for multiperiod nonlinear stochastic programming models.The intention of the research program is to build  realistic models for dynamic control under uncertainty and to study efficient solution algorithms for theoretically intractable problems.