Assessing calibration of multinomial risk prediction models
Assessing calibration of multinomial risk prediction models
复制标题
DOI：
10.1002/sim.6114
复制
发表时间：
2014-07-10
影响因子：
2
通讯作者：
Van Calster, Ben
中科院分区：
文献类型：
--
作者：
Van Hoorde, Kirsten;Vergouwe, Yvonne;Van Calster, Ben
校准，即观察到的结果是否与预测的风险一致，在评估风险预测模型时很重要。对于二分结果，存在几种工具来评估模型校准的不同方面，例如大规模校准、逻辑重新校准和（非）参数校准图。我们的目标是将这些工具扩展到多分类结局的预测模型。我们专注于使用多项逻辑回归（MLR）开发的模型：使用k-1个方程预测具有k个类别的结局Y，比较每个类别i（i=2，...，k）使用一组预测器与参考类别1进行比较，得到k-1个线性预测器。我们提出了一个多项逻辑重新校准框架，涉及MLR拟合，其中Y是使用预测模型中的k-1个线性预测因子预测的。非参数替代方案可以使用向量样条用于线性预测因子的效应。参数和非参数框架可用于生成多项式校准图。此外，参数框架可用于估计和校准截距和slopes.Two的统计检验说明性的案例研究，一个卵巢肿瘤的恶性诊断和残留的质量诊断睾丸癌患者顺铂为基础的化疗。风险预测模型分别基于2037例和544例患者的数据开发，并分别在1107例和550例患者上进行了外部验证。多分支校准图通过校准性能的视觉总结提供了特别丰富的信息。版权所有（c）2014约翰威利父子有限公司
Calibration, that is, whether observed outcomes agree with predicted risks, is important when evaluating risk prediction models. For dichotomous outcomes, several tools exist to assess different aspects of model calibration, such as calibration-in-the-large, logistic recalibration, and (non-)parametric calibration plots. We aim to extend these tools to prediction models for polytomous outcomes.We focus on models developed using multinomial logistic regression (MLR): outcome Y with k categories is predicted using k-1 equations comparing each category i (i=2,...,k) with reference category 1 using a set of predictors, resulting in k-1 linear predictors. We propose a multinomial logistic recalibration framework that involves an MLR fit where Y is predicted using the k-1 linear predictors from the prediction model. A non-parametric alternative may use vector splines for the effects of the linear predictors. The parametric and non-parametric frameworks can be used to generate multinomial calibration plots. Further, the parametric framework can be used for the estimation and statistical testing of calibration intercepts and slopes.Two illustrative case studies are presented, one on the diagnosis of malignancy of ovarian tumors and one on residual mass diagnosis in testicular cancer patients treated with cisplatin-based chemotherapy. The risk prediction models were developed on data from 2037 and 544 patients and externally validated on 1107 and 550 patients, respectively.We conclude that calibration tools can be extended to polytomous outcomes. The polytomous calibration plots are particularly informative through the visual summary of the calibration performance. Copyright (c) 2014 John Wiley & Sons, Ltd.