Developing clinical prediction models when adhering to minimum sample size recommendations: The importance of quantifying bootstrap variability in tuning parameters and predictive performance.
Developing clinical prediction models when adhering to minimum sample size recommendations: The importance of quantifying bootstrap variability in tuning parameters and predictive performance.
复制标题
DOI：
10.1177/09622802211046388
复制
发表时间：
2021-12
影响因子：
2.3
通讯作者：
Sperrin M
中科院分区：
文献类型：
--
作者：
Martin GP;Riley RD;Collins GS;Sperrin M
最近的最小样本量公式（Riley等人）用于开发临床预测模型有助于确保开发数据集具有足够的大小以最小化过拟合。虽然已知这些标准可以避免平均过度拟合，但在推荐样本量下过度拟合的变异程度尚不清楚。我们调查了这一点，通过模拟研究和实证的例子，开发逻辑回归临床预测模型，使用未惩罚的最大似然估计，和各种后估计收缩或惩罚方法。虽然平均校准斜率接近所有方法的理想值1，但平均而言，与未惩罚的方法相比，惩罚进一步降低了过拟合水平。这是以外部数据中惩罚方法的预测性能的更高可变性为代价的。我们建议在满足或超过最小样本量要求的数据中使用惩罚方法，以进一步减轻过拟合，并且应始终将预测性能和任何调整参数的可变性作为模型开发过程的一部分进行检查，因为这提供了单独的平均（优化调整）性能的额外信息。较低的变异性将保证所开发的临床预测模型将在来自与用于模型开发的相同群体的新个体中表现良好。
Recent minimum sample size formula (Riley et al.) for developing clinical prediction models help ensure that development datasets are of sufficient size to minimise overfitting. While these criteria are known to avoid excessive overfitting on average, the extent of variability in overfitting at recommended sample sizes is unknown. We investigated this through a simulation study and empirical example to develop logistic regression clinical prediction models using unpenalised maximum likelihood estimation, and various post-estimation shrinkage or penalisation methods. While the mean calibration slope was close to the ideal value of one for all methods, penalisation further reduced the level of overfitting, on average, compared to unpenalised methods. This came at the cost of higher variability in predictive performance for penalisation methods in external data. We recommend that penalisation methods are used in data that meet, or surpass, minimum sample size requirements to further mitigate overfitting, and that the variability in predictive performance and any tuning parameters should always be examined as part of the model development process, since this provides additional information over average (optimism-adjusted) performance alone. Lower variability would give reassurance that the developed clinical prediction model will perform well in new individuals from the same population as was used for model development.
登录
查看更多内容
回顾和评估用于事件少的低维数据风险预测的惩罚回归方法。
影响因子：
2
作者：
Pavlou M;Ambler G;Seaman S;De Iorio M;Omar RZ
通讯作者：
Omar RZ
影响因子：
7.2
作者：
Riley RD;Snell KIE;Martin GP;Whittle R;Archer L;Sperrin M;Collins GS
通讯作者：
Collins GS
影响因子：
2.7
作者：
FIRTH, D
通讯作者：
FIRTH, D
影响因子：
2
作者：
Puhr, Rainer;Heinze, Georg;Geroldinger, Angelika
通讯作者：
Geroldinger, Angelika
影响因子：
2.5
作者：
HOERL, AE;KENNARD, RW
通讯作者：
KENNARD, RW