Data-driven stochastic dynamic programming approaches for optimal planning of disease screening and chronic disorder management

在向基于证据和个性化决策转变的情况下，需要强大的数据驱动建模方法来获得针对实际工程问题的个性化最优干预方案。我们建议开发基于随机动态规划和数据分析的新的工程工具来解决实际的序列决策问题。虽然已提出的研究将应用于癌症筛查和慢性病管理，但重点是开发工程方法。*首先，我们提出了一种新的双准则部分可观测马尔可夫决策过程(POMDP)来推导特定筛查问题的最优Pareto有效策略。关于多准则POMDP的研究很有限，现有的方法都是基于状态离散化和机器学习的近似方法。为了准确地求解POMDP，我们提出了一种新的模型形式，通过用有限的历史筛选观测集合来代替状态空间，将模型表示为约束马尔可夫决策过程(MDP)。在合理的条件下，这种重新表述提供了一个易于处理的模型，我们的目标是通过结构性质减少状态/动作空间，为其开发有效的求解程序。使用文献和罗切斯特-明尼苏达州梅奥诊所的临床数据，我们将应用该配方来确定结直肠癌筛查和监测的帕累托有效政策。拟议的模型可能有助于开发洞察力，以改进癌症筛查实践并使其个性化，这是一个重大贡献，因为癌症是加拿大的主要死亡原因。该方法还将为其他工程应用提高POMDP的工程知识。*第二，我们提出了一种新的数据驱动方法，用于对不可逆恶化的系统(例如，慢性病)及其管理过程进行建模。许多系统由复杂的评分系统根据测试/检查结果或技术评估的分数进行监测。拟议的数据分析方法将处理来自检查的纵向记录的数据，以确定关键事件(收费站)，并通过这些收费站使用包括监督机器学习、分类和预测模型在内的方法估计未来恶化的进展。这些建议的说明性/预测性分析工具的研究结果将被用于推导MDP模型，以优化姑息性/辅助性干预的时机，以减少因病情恶化而造成的不良影响。建议的方法将应用于预测肌萎缩侧索硬化症(ALS)的进展，优化订购辅助设备的时机，并利用罗切斯特梅奥诊所的数据最大化ALS患者的福祉。*建议的研究将扩展到考虑更一般的环境。*

Given the transformation towards evidence-based and personalized decision making, powerful data-driven modeling approaches are needed to obtain personalized optimal intervention plans for practical engineering problems. We propose developing novel engineering tools based on stochastic dynamic programming and data analytics to address practical sequential decision making problems. Although proposed research will be applied to cancer screening and chronic-disease management, the focus is on developing engineering methodology.*******First, we propose a novel bicriteria partially observable Markov decision process (POMDP) to derive the optimal Pareto-efficient policies for particular screening problems. There is limited research on multicriteria POMDPs; the existing approaches are approximations based on methods including state discretization and machine learning. To solve our POMDP exactly, we propose a novel reformulation of the model as a constrained Markov decision process (MDP) by replacing the state space with a limited collection of historical screening observations. Under reasonable conditions, this reformulation provides a tractable model for which we aim to develop efficient solution procedures by reducing the state/action space via structural properties. Using clinical data from literature and Mayo Clinic, Rochester-MN, we will apply this formulation to determine the Pareto-efficient policies for colorectal cancer screening and surveillance. The proposed model may help develop insights to improve and personalize cancer screening practices, a significant contribution as cancer is the leading cause of death in Canada. The proposed approach will also improve the engineering knowledge on POMDPs for other engineering applications.*******Second, we propose a novel data-driven approach for modeling the progression of irreversibly deteriorating systems (e.g., chronic diseases) and their management. Many systems are monitored by complex scoring systems based on test/inspection results or scores from technical assessments. The proposed data analytics approach will process data from longitudinal records of inspections to identify critical events (tollgates) and estimate future progression of deterioration through these tollgates, by using methods including supervised machine learning, classification, and prediction models. The findings from these proposed prescriptive/predictive analytics tools will then be used to derive an MDP model to optimize the timing of palliative/assistive interventions to decrease the disutility due to deterioration. The proposed methodology will be applied to predict amyotrophic lateral sclerosis (ALS) progression, optimize the timing of ordering assistive devices, and maximize wellbeing of ALS patients using data from Mayo Clinic, Rochester.*******The proposed research will be extended to consider more general settings.***