Prediction models need appropriate internal, internal-external, and external validation.
Prediction models need appropriate internal, internal-external, and external validation.
复制标题
DOI：
10.1016/j.jclinepi.2015.04.005
复制
发表时间：
2016-01
影响因子：
7.2
通讯作者：
Harrell FE Jr
中科院分区：
文献类型：
--
作者：
Steyerberg EW;Harrell FE Jr
最近的另一场辩论是关于对已发表的新预测模型缺乏外部验证研究的解释[3][4][5]。一个问题是验证在模型开发时应该扮演的角色。许多研究人员可能会试图报告一些外部有效性的证据，即关于歧视和校准，在独立样本中，他们的出版物提出了一个新的预测模型。目前主要的临床期刊似乎都很欣赏这种报道。另一个问题是外部验证是否应该由与参与预测模型开发的作者不同的作者进行[3][6]。我们想对预测建模的科学基础中的这些和相关的关键问题进行评论。最近的综述证实，对于指定预测模型的形式（包括哪些预测因子）和预测因子效应的估计（标准估计方法的过拟合）所带来的复杂挑战，模型开发研究通常相对较小[3]。中位样本量为445例受试者。事件的数量是这类研究的限制因素，对于可靠的建模来说可能太低[4]。在如此小的样本中，内部验证是必不可少的，并且明显的性能估计非常乐观（图1）。Bootstrapping是验证预测模型的首选方法[7][8][9]。引导程序应该包括所有建模步骤，以诚实地评估模型性能[10]。特别是任何模型选择步骤，如变量选择，需要重复每个自助样本（如果使用）。
Another recent debate was on the interpretation of the lack of external validation studies of published novel prediction models [3][4][5]. One issue is the role that validation should have at the time of model development. Many researchers may be tempted to try to report some proof for external validity, ie on discrimination and calibration, in independent samples with their publication that proposes a new prediction model. Major clinical journals currently seem to appreciate such reporting. Another issue is whether external validation should be performed by different authors than those involved in the development of the prediction model [3][6]. We would like to comment on these and related key issues in the scientific basis of prediction modeling.The recent review confirms that model development studies are often relatively small for the complex challenges posed by specifying the form of a prediction model (which predictors to include) and the estimation of predictor effects (overfit with standard estimation methods)[3]. The median sample size was 445 subjects. The number of events is the limiting factor in this type of research and may be far too low for reliable modeling [4]. In such small samples, internal validation is essential, and apparent performance estimates are severely optimistic (Figure 1). Bootstrapping is the preferred approach for validation of prediction models [7][8][9]. A bootstrap procedure should include all modeling steps for an honest assessment of model performance [10]. Specifically any model selection steps, such as variable selection, need to be repeated per bootstrap sample if used.
登录
查看更多内容
影响因子：
15.8
作者：
Steyerberg EW;Mushkudiani N;Perel P;Butcher I;Lu J;McHugh GS;Murray GD;Marmarou A;Roberts I;Habbema JD;Maas AI
通讯作者：
Maas AI
影响因子：
120.7
作者：
Balmana, Judith;Stockwell, David H.;Syngal, Sapna
通讯作者：
Syngal, Sapna
影响因子：
39.2
作者：
Justice, AC;Covinsky, KE;Berlin, JA
通讯作者：
Berlin, JA
影响因子：
39.2
作者：
Collins, Gary S.;Reitsma, Johannes B.;Moons, Karel G. M.
通讯作者：
Moons, Karel G. M.
影响因子：
7.2
作者：
Debray, Thomas P. A.;Vergouwe, Yvonne;Moons, Karel G. M.
通讯作者：
Moons, Karel G. M.