Markov decision processes: a tool for sequential decision making under uncertainty.
Markov decision processes: a tool for sequential decision making under uncertainty.
复制标题
DOI：
10.1177/0272989x09353194
复制
发表时间：
2010-07
期刊：
影响因子：
--
通讯作者：
Roberts MS
中科院分区：
文献类型：
--
作者：
Alagoz O;Hsu H;Schaefer AJ;Roberts MS
我们提供了一个教程的马尔可夫决策过程（MDP），这是强大的分析工具，用于顺序决策下的不确定性，已被广泛用于许多工业和制造业的应用，但在医疗决策（MDM）的建设和评估。我们演示了使用MDP解决不确定性下的顺序临床治疗问题。马尔可夫决策过程一般化标准马尔可夫模型，因为决策过程嵌入模型中，并且随着时间的推移做出多个决策。此外，它们比标准决策分析具有显着优势。我们比较MDP标准马尔可夫为基础的模拟模型，通过解决问题的最佳时机活体肝移植使用这两种方法。这两种模型导致相同的最佳移植政策和相同的总预期寿命为同一患者和活体供体。求解MDP模型的计算时间明显小于求解马尔可夫模型的计算时间。我们简要介绍了越来越多的文献的MDPs应用于医疗决策。
We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.