A Bayesian Perspective on Early Stage Event Prediction in Longitudinal Data
A Bayesian Perspective on Early Stage Event Prediction in Longitudinal Data
复制标题
DOI：
10.1109/tkde.2016.2608347
复制
发表时间：
2016-12-01
影响因子：
8.9
通讯作者：
Reddy, Chandan K.
中科院分区：
文献类型：
--
作者：
Fard, Mahtab Jahanbani;Wang, Ping;Reddy, Chandan K.
在纵向研究的早期阶段预测事件发生是一个重要且具有挑战性的问题，在许多实际应用中具有很高的实用价值。与领域专家可以在合理的短时间内为数据提供标签的标准分类和回归问题相反，这种纵向研究中的训练数据必须仅通过等待足够数量的事件的发生来获得。生存分析的目的是使用过去在一定时间内收集的数据直接预测关注事件的时间。但是，它不能回答“如何利用研究早期其他受试者的事件发生信息预测受试者是否会在纵向研究结束时发生事件”的开放性问题。".这项工作的目标是预测事件发生在未来的时间点，只使用有关的信息发生在纵向研究的初始阶段的有限数量的事件。该问题表现出两个主要挑战：（1）缺乏关于事件发生的完整信息（删失）和（2）仅可获得研究初始阶段发生的部分事件集。我们提出了一种新的早期预测（ESP）框架，用于构建在纵向研究早期阶段训练的事件预测模型。首先，我们开发了一种新的方法来解决第一个挑战，通过引入一种新的方法来处理删失数据使用Kaplan-Meier估计。然后，我们扩展了朴素贝叶斯，树增强朴素贝叶斯（TAN），贝叶斯网络方法的基础上提出的框架，并开发了三种算法，即ESP-NB，ESP-TAN，ESPBN，有效地预测事件的发生使用训练数据在研究的早期阶段。更具体地说，我们的方法有效地集成了贝叶斯方法与加速故障时间（AFT）模型，通过适应未来时间点的事件发生的先验概率。使用广泛的合成和真实世界的基准数据集评估所提出的框架。我们广泛的实验表明，当训练数据中仅使用有限的事件信息时，与现有方案相比，所提出的ESP框架平均准确率高出20%。
Predicting event occurrence at the early stage of a longitudinal study is an important and challenging problem which has high practical value in many real-world applications. As opposed to the standard classification and regression problems where a domain expert can provide labels for the data in a reasonably short period of time, training data in such longitudinal studies must be obtained only by waiting for the occurrence of a sufficient number of events. Survival analysis aims at directly predicting the time to an event of interest using the data collected in the past for a certain duration. However, it cannot give an answer to the open question of "how to forecast whether a subject will experience an event by end of a longitudinal study using event occurrence information of other subjects at the early stage of the study?". The goal of this work is to predict the event occurrence at a future time point using only the information about a limited number of events that occurred at the initial stages of a longitudinal study. This problem exhibits two major challenges: (1) absence of complete information about event occurrence (censoring) and (2) availability of only a partial set of events that occurred during the initial phase of the study. We propose a novel Early Stage Prediction (ESP) framework for building event prediction models which are trained at the early stages of longitudinal studies. First, we develop a novel approach to address the first challenge by introducing a new method for handling censored data using Kaplan-Meier estimator. We then extend the Naive Bayes, Tree-Augmented Naive Bayes (TAN), and Bayesian Network methods based on the proposed framework, and develop three algorithms, namely, ESP-NB, ESP-TAN, and ESPBN, to effectively predict event occurrence using training data obtained at an early stage of the study. More specifically, our approach effectively integrates Bayesian methods with an Accelerated Failure Time (AFT) model by adapting the prior probability of the event occurrence for future time points. The proposed framework is evaluated using a wide range of synthetic and real-world benchmark datasets. Our extensive set of experiments show that the proposed ESP framework is, on an average, 20 percent more accurate compared to existing schemes when using only limited event information in the training data.