Dynamic treatment regimes via smooth surrogate loss: theory, methods, and computational aspects

在癌症、败血症或抑郁症等复杂疾病中，由于疾病的动态性质，患者通常需要在多个阶段进行治疗。在这种情况下，医生可能需要一个算法或政策，以便在必要时切换到替代治疗选项，除了开出最初的治疗方案外。此外，由于患者对治疗的反应不同，医生需要根据每个患者的特定需求和概况来个性化治疗策略。该项目旨在利用现代灵活的机器学习技术和现有的患者数据来确定在这种时变情况下的最佳治疗策略，即最优动态治疗方案(DTR)。确保具有大样本容量的真实世界电子健康记录数据的可扩展性将是一个关键关注点。该项目还将开发和分发用户友好的开源软件，并为研究生提供研究培训经验。最近的研究将DTR政策学习与顺序分类问题联系起来，使机器学习技术能够集成。然而，目前，用于解决所得到的分类问题的计算高效的方法仅限于特定情况，例如二元处理设置，并且容易出现方差膨胀。该项目的第一步旨在证明，任何DTR策略学习问题中的潜在顺序分类都可以通过平滑代理问题得到证明。这种代理问题将服从于可扩展的机器学习工具，例如随机梯度下降，从而促进快速实现。在第二步中，本项目将结合上述基于机器学习的方法和基于模型的技术来构建更稳定的混合估计器，该估计器对潜在的模型错误指定具有鲁棒性。此外，该项目将通过整合分类理论、离线强化学习和非参数统计推断理论来全面调查产生的方法的性能限制。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In complex diseases such as cancer, sepsis, or depression, patients often require treatments in multiple stages due to the dynamic nature of the disease. In such cases, physicians may need an algorithm or policy to switch to an alternative treatment option when necessary, in addition to prescribing the initial treatment. Furthermore, since patients respond differently to treatments, physicians need to personalize the treatment policy based on each patient's specific needs and profile. This project aims to utilize modern, flexible machine learning techniques and existing patient data to identify the optimal treatment policy, known as the optimal dynamic treatment regime (DTR), in such time-varying situations. Ensuring scalability to real-world electronic health record data with a large sample size will be a key focus. This project will also develop and distribute user-friendly open-source software and provide research training experiences for graduate students.Recent research has connected DTR policy learning to sequential classification problems, enabling the integration of machine learning techniques. However, currently, computationally efficient methods for solving the resulting classification problems are limited to specific cases, such as binary-treatment settings, and are prone to variance inflation. The first step of this project aims to demonstrate that the underlying sequential classification in any DTR policy learning problem can be provably solved via a smooth surrogate problem. This surrogate problem will be amenable to scalable machine-learning tools, such as stochastic gradient descent, facilitating fast implementation. In the second step, this project will combine the aforementioned machine-learning-based method with model-based techniques to construct a more stable hybrid estimator that is robust to potential model misspecifications. In addition, this project will comprehensively investigate the performance limits of the resulting methods by integrating classification theory, offline reinforcement learning, and the theory of nonparametric statistical inference.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.