Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond
Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond
复制标题
DOI：
10.1002/sim.2929
复制
发表时间：
2008-01-30
影响因子：
2
通讯作者：
Vasan, Ramachandran S.
中科院分区：
文献类型：
--
作者：
Pencina, Michael J.;D'Agostino, Ralph B., Sr.;Vasan, Ramachandran S.
识别与发生心血管疾病风险相关的关键因素，并使用多变量预测算法量化这种风险，是20世纪预防心脏病学和心血管流行病学取得的主要进展之一。科学家正在不断发现新的风险标记物，这为统计学家和临床医生评估这些生物标记物并开发包含它们的新风险配方带来了机遇和挑战。关键问题之一是如何最好地评估和量化这些新模型在风险预测方面的改进。仅证明一种新的生物标记物与心血管风险之间具有统计学意义的相关性是不够的。一些研究人员提出，应以接收者-操作特征曲线(AUC)下面积的改善为主要标准，而另一些研究人员则认为需要更好地衡量预测模型的性能。在这篇文章中，我们通过引入两个新的衡量标准来解决这个问题，一个基于综合的敏感性和特异性，另一个基于重分类表格。这些新措施提供了有关AUC的增量信息。我们讨论了这些新措施的性质，并将它们与AUC进行了比较。我们还发展了简单的有意义的渐近检验。我们用弗雷明翰心脏研究中的一个例子来说明这些措施的使用。我们建议科学家在评估新的生物标记物的性能时，除了AUC之外，还要考虑这些类型的测量。版权所有(C)2007 John Wiley&Sons，Ltd.
Identification of key factors associated with the risk of developing cardiovascular disease and quantification of this risk using multivariable prediction algorithms are among the major advances made in preventive cardiology and cardiovascular epidemiology in the 20th century. The ongoing discovery of new risk markers by scientists presents opportunities and challenges for statisticians and clinicians to evaluate these biomarkers and to develop new risk formulations that incorporate them. One of the key questions is how best to assess and quantify the improvement in risk prediction offered by these new models. Demonstration of a statistically significant association of a new biomarker with cardiovascular risk is not enough. Some researchers have advanced that the improvement in the area under the receiver-operating-characteristic curve (AUC) should be the main criterion, whereas others argue that better measures of performance of prediction models are needed. In this paper, we address this question by introducing two new measures, one based on integrated sensitivity and specificity and the other on reclassification tables. These new measures offer incremental information over the AUC. We discuss the properties of these new measures and contrast them with the AUC. We also develop simple asymptotic tests of significance. We illustrate the use of these measures with an example from the Framingham Heart Study. We propose that scientists consider these types of measures in addition to the AUC when assessing the performance of newer biomarkers. Copyright (c) 2007 John Wiley & Sons, Ltd.