Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis
Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis
复制标题
DOI：
10.1016/s0895-4356(01)00341-9
复制
发表时间：
2001-08-01
影响因子：
7.2
通讯作者：
Habbema, JDF
中科院分区：
文献类型：
--
作者：
Steyerberg, EW;Harrell, FE;Habbema, JDF
预测模型的性能被高估时，简单地确定用于构建模型的受试者的样本。有几种内部验证方法可用于提供新主题中模型性能的更准确估计。我们评估了几种不同的分裂样本，交叉验证和自举方法与逻辑回归模型，其中包括8个预测急性心肌梗死后30天死亡率。从大型数据集（GUSTO-I; n = 40，830; 2851例死亡）中抽取随机样本，样本量在n = 572和n = 9165之间，以反映每个变量5 - 80起事件的数据集建模。确定其余受试者的独立表现。性能指标包括辨别能力，校准和整体准确性。我们发现，分离样本分析对性能的估计过于悲观，且变异性很大。对10%样本的交叉验证具有低偏倚和低变异性，但不适用于所有性能指标。内部效度可以最好地估计与Bootstrapping，它提供了稳定的估计与低偏差。我们的结论是，分裂样本验证是低效的，并建议自举估计预测逻辑回归模型的内部有效性。(C)2001 Elsevier Science Inc. All rights reserved.
The performance of a predictive model is overestimated when simply determined on the sample of subjects that was used to construct the model. Several internal validation methods are available that aim to provide a more accurate estimate of model performance in new subjects. We evaluated several variants of split-sample, cross-validation and bootstrapping methods with a logistic regression model that included eight predictors for 30-day mortality after an acute myocardial infarction. Random samples with a size between,n = 572 and n = 9165 were drawn from a large data set (GUSTO-I; n = 40,830; 2851 deaths) to reflect modeling in data sets with between 5 and 80 events per variable. Independent performance was determined on the remaining subjects. Performance measures included discriminative ability, calibration and overall accuracy. We found that split-sample analyses gave overly pessimistic estimates of performance with large variability. Cross-validation on 10% of the sample had low bias and low variability, but was not suitable fur all performance measures. Internal validity could best be estimated with bootstrapping, which provided stable estimates with low bias. We conclude that split-sample validation is inefficient, and recommend bootstrapping for estimation of internal validity of a predictive logistic regression model. (C) 2001 Elsevier Science Inc. All rights reserved.