The Net Reclassification Index (NRI): a Misleading Measure of Prediction Improvement Even with Independent Test Data Sets.
The Net Reclassification Index (NRI): a Misleading Measure of Prediction Improvement Even with Independent Test Data Sets.
复制标题
DOI：
10.1007/s12561-014-9118-0
复制
发表时间：
2015-10-01
影响因子：
1
通讯作者：
Hilden J
中科院分区：
文献类型：
--
作者：
Pepe MS;Fan J;Feng Z;Gerds T;Hilden J
净重分类指数(NRI)是一种非常流行的衡量标准，用来评估通过在一组基线预测指标中增加一个标记而获得的预测性能的改善。然而，这一新的衡量标准的统计特性还没有得到深入的探讨。我们证明了一个令人震惊的结果，即使当新的标记没有预测性信息时，使用来自训练集的风险模型在大型测试数据集上计算的NRI统计也可能是正的。提供了一个相关的理论例子，其中包括非信息性标记的错误风险函数被证明错误地产生正的NRI。对这一现象提供了一些见解。由于NRI统计量的较大值可能只是由于使用了不适合的风险模型，因此我们建议谨慎使用NRI作为标记评估的基础。其他预测绩效改进的衡量标准，如从ROC曲线、净收益函数和Brier评分得出的衡量标准，由于风险函数拟合不佳，不可能很大。
The Net Reclassification Index (NRI) is a very popular measure for evaluating the improvement in prediction performance gained by adding a marker to a set of baseline predictors. However, the statistical properties of this novel measure have not been explored in depth. We demonstrate the alarming result that the NRI statistic calculated on a large test dataset using risk models derived from a training set is likely to be positive even when the new marker has no predictive information. A related theoretical example is provided in which an incorrect risk function that includes an uninformative marker is proven to erroneously yield a positive NRI. Some insight into this phenomenon is provided. Since large values for the NRI statistic may simply be due to use of poorly fitting risk models, we suggest caution in using the NRI as the basis for marker evaluation. Other measures of prediction performance improvement, such as measures derived from the ROC curve, the net benefit function and the Brier score, cannot be large due to poorly fitting risk functions.