Evaluation of polygenic risk models using multiple performance measures: a critical assessment of discordant results.
Evaluation of polygenic risk models using multiple performance measures: a critical assessment of discordant results.
复制标题
DOI：
10.1038/s41436-018-0058-9
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Janssens ACJW
中科院分区：
文献类型：
--
作者：
Martens FK;Tonk ECM;Janssens ACJW
受试者工作特征曲线下面积（AUC）通常用于评估多基因风险模型的改善，并越来越多地与净重新分类改善（NRI）和综合区分改善（IDI）一起评估。我们评估了研究人员在同时评估时如何描述和解释AUC，NRI和IDI。我们回顾了研究人员如何描述AUC，NRI和IDI的定义以及他们如何计算每个指标。接下来，我们回顾了如何解释AUC、NRI和IDI的增量;以及如何得出关于风险模型改进的总体结论。AUC、NRI和IDI在63%、70%和0%的文章中被正确定义。所有统计学显著值和几乎一半的非显著值均被解释为改善的指标，无论指标的值如何。此外，当NRI和IDI具有统计学显著性时，AUC的微小、非显著性变化被解释为改善的迹象。研究人员对如何解释多基因风险模型预测性能评估的各种指标的知识不足，并依赖于其解释的统计学意义。需要更好的理解，以实现更有意义的解释多基因预测研究。
The area under the receiver operating characteristic curve (AUC) is commonly used for evaluating the improvement of polygenic risk models and increasingly assessed together with the net reclassification improvement (NRI) and integrated discrimination improvement (IDI). We evaluated how researchers described and interpreted AUC, NRI, and IDI when simultaneously assessed. We reviewed how researchers described definitions of AUC, NRI and IDI and how they computed each metric. Next, we reviewed how the increment in AUC, NRI and IDI were interpreted; and how the overall conclusion about the improvement of the risk model was reached. AUC, NRI and IDI were correctly defined in 63%, 70%, and 0% of the articles. All statistically significant values and almost half of the non-significant were interpreted as indicative of improvement, irrespective of the values of the metrics. Also, small, nonsignificant changes in the AUC were interpreted as indication of improvement when NRI and IDI were statistically significant. Researchers have insufficient knowledge about how to interpret the various metrics for the assessment of the predictive performance of polygenic risk models and rely on the statistical significance for their interpretation. A better understanding is needed to achieve more meaningful interpretation of polygenic prediction studies.
登录
查看更多内容
影响因子：
5
作者：
Mihaescu, Raluca;van Zitteren, Moniek;Janssens, A. Cecile J. W.
通讯作者：
Janssens, A. Cecile J. W.
影响因子：
2
作者：
Pencina, Michael J.;D'Agostino, Ralph B., Sr.;Steyerberg, Ewout W.
通讯作者：
Steyerberg, Ewout W.
影响因子：
5
作者：
Pepe, MS;Janes, H;Newcomb, P
通讯作者：
Newcomb, P
影响因子：
37.8
作者：
Cook, Nancy R.
通讯作者：
Cook, Nancy R.
影响因子：
7.2
作者：
Janssens, A Cecile J W;Ioannidis, John P A;Khoury, Muin J
通讯作者：
Khoury, Muin J