Q-LEARNING WITH CENSORED DATA.
Q-LEARNING WITH CENSORED DATA.
复制标题
Q学习数据。
DOI：
10.1214/12-aos968
复制
发表时间：
2012-02-01
影响因子：
4.5
通讯作者：
Kosorok MR
中科院分区：
文献类型：
--
作者：
Goldberg Y;Kosorok MR
我们发展了一个多阶段决策问题的方法论，该问题具有灵活的阶段数，其中的回报是受审查的生存时间。我们提出了一种新的Q学习算法，该算法对删失数据进行调整，并允许灵活的阶段数。给出了该算法学习的策略的泛化误差的有限样本界，并证明了当最优Q-函数属于逼近空间时，该算法得到的策略的期望生存时间收敛于最优策略的期望生存时间。我们模拟了一个具有可变阶段数的多阶段临床试验，并应用所提出的删失Q学习算法来寻找个性化的治疗方案。本文提出的方法论对癌症和其他危及生命的疾病的个性化药物试验的设计具有一定的指导意义。
We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases.