Reporting and methods in clinical prediction research: a systematic review.
Reporting and methods in clinical prediction research: a systematic review.
复制标题
DOI：
10.1371/journal.pmed.1001221
复制
发表时间：
2012
期刊：
影响因子：
15.8
通讯作者：
Moons KG
中科院分区：
文献类型：
--
作者：
Bouwmeester W;Zuithoff NP;Mallett S;Geerlings MI;Vergouwe Y;Steyerberg EW;Altman DG;Moons KG
Walter Bouwmeester和他的同事在2008年调查了六份高影响力的普通医学期刊上预测研究的报告和方法，发现大多数预测研究没有遵循当前的方法建议。我们调查了预测研究的报告和方法，重点是目标、设计、参与者选择、结果、预测因子、统计能力、统计方法和预测性能指标。我们使用全手检索来确定2008年在六个高影响力的普通医学期刊上发表的所有预测研究。我们根据最近对预测研究的建议，制定了一个全面的项目清单，系统地对研究的行为和报告进行评分。两名评论者独立对研究进行评分。我们检索了71篇论文进行全文综述：51篇是预测因子发现研究，14篇是预测模型开发研究，3篇是对先前开发的模型进行外部验证，3篇报告了模型对参与者结果的影响。15%的研究设计不明确，大多数研究（60%）采用前瞻性队列。对参与者的描述以及预测因子和结果的定义总体上是好的。尽管许多人反对这样做，但连续预测因子经常被一分为二（32%的研究）。在67%的研究中，无法确定每个预测因子的事件数作为统计能力的衡量标准；在剩下的人中，53%的人少于通常推荐的每个预测器10个事件的值。大多数研究（68%）描述了候选预测因子的先验选择方法。在多变量分析中，大量研究依赖于p<0.05的p值截断值来选择预测因子（29%）。分别有12%和27%的研究报告了预测模型性能度量，即校准和判别。高影响力期刊上的大多数预测研究没有遵循当前的方法建议，限制了它们的可靠性和适用性。在我们的生活中，我们常常希望能够预测未来。例如，股票市场会上涨吗？或者明天会下雨吗？对病人和医生来说，能够预测未来的健康状况也很重要，而且发表的临床“预测研究”越来越多。诊断预测研究调查变量或测试结果预测特定诊断存在或不存在的能力。例如，最近的一项研究比较了两种成像技术诊断肺栓塞（肺中的血块）的能力。预后预测研究调查了各种标志物预测未来结果的能力，如心脏病发作的风险。两种类型的预测研究都可以调查患者特征、单变量、测试或标记，或变量、测试或标记的组合（多变量研究）的预测特性。这两种类型的预测研究还可以包括建立多变量预测模型来指导患者管理（模型开发），或测试模型的性能（验证），或量化使用预测模型对患者和医生行为和结果的影响（影响评估）的研究。随着预测研究的增加，人们对这类研究方法的兴趣越来越大，因为做得不好或报告得不好的预测研究可能具有有限的可靠性和适用性，因此在患者管理中几乎没有用处。在本系统综述中，研究人员通过检查目标、设计、参与者选择、结果和候选预测因子的定义和测量、统计能力和分析以及包括在2008年发表在几家普通医学期刊上的多变量预测研究文章中的性能测量来调查预测研究的报告和方法。在系统综述中，研究人员使用一套预定义的标准确定对给定主题进行的所有研究，并系统地分析这些研究的报告方法和结果。研究人员通过浏览所有期刊（手动搜索），确定了2008年发表在六份高影响力普通医学期刊上的所有符合其预定义标准的多变量预测研究。然后，他们根据预测研究的最新建议（例如，肿瘤标志物预后研究的报告建议- REMARK指南），使用综合项目列表对每个研究的方法和报告进行评分。在检索到的71项研究中，51项是预测因子发现研究，14项是预测模型开发研究，3项外部验证了现有模型，3项报告了模型对参与者结果的影响。研究设计、参与者选择、结果和预测因素的定义以及预测因素的选择通常都得到了很好的报道，但研究的其他方法学和报告方面并不理想。例如，尽管有许多建议，但连续预测器通常是二分的。也就是说，不是在预测模型中使用变量的测量值（例如，在心血管疾病预测模型中使用血压），而是经常将测量值分配给两大类。同样，许多研究未能充分估计最小化预测效应偏差所需的样本量，并且很少有模型开发论文量化并验证了所提出模型的预测性能。这些发现表明，在2008年，发表在高影响力的普通医学期刊上的大多数预测研究都没有遵循现行的临床预测研究进行和报告指南。由于本研究的研究发表在高影响力的医学期刊上，因此它们很可能是2008年发表的高质量研究的代表。然而，自2008年以来，报告标准可能有所改善，预测研究的实施实际上可能比这一分析所表明的要好，因为通常应用于期刊文章的长度限制可能解释了一些报告遗漏。然而，尽管有一些令人鼓舞的发现，研究人员得出结论，他们在许多已发表的预测研究中发现的不良报告和不良方法令人担忧，并可能限制这类临床研究的可靠性和适用性。请通过本摘要的在线版本（http://dx.doi.org/10.1371/journal.pmed.1001221）访问这些网站。EQUATOR网络是一项国际倡议，旨在通过促进透明和准确的研究报告，提高医学研究文献的可靠性和价值；Cochrane预后方法组（Cochrane Prognosis Methods Group）提供了更多关于预后研究方法的信息。Cochrane预后方法组提供了更多关于预后研究方法的信息
Walter Bouwmeester and colleagues investigated the reporting and methods of prediction studies in 2008, in six high-impact general medical journals, and found that the majority of prediction studies do not follow current methodological recommendations. We investigated the reporting and methods of prediction studies, focusing on aims, designs, participant selection, outcomes, predictors, statistical power, statistical methods, and predictive performance measures. We used a full hand search to identify all prediction studies published in 2008 in six high impact general medical journals. We developed a comprehensive item list to systematically score conduct and reporting of the studies, based on recent recommendations for prediction research. Two reviewers independently scored the studies. We retrieved 71 papers for full text review: 51 were predictor finding studies, 14 were prediction model development studies, three addressed an external validation of a previously developed model, and three reported on a model's impact on participant outcome. Study design was unclear in 15% of studies, and a prospective cohort was used in most studies (60%). Descriptions of the participants and definitions of predictor and outcome were generally good. Despite many recommendations against doing so, continuous predictors were often dichotomized (32% of studies). The number of events per predictor as a measure of statistical power could not be determined in 67% of the studies; of the remainder, 53% had fewer than the commonly recommended value of ten events per predictor. Methods for a priori selection of candidate predictors were described in most studies (68%). A substantial number of studies relied on a p-value cut-off of p<0.05 to select predictors in the multivariable analyses (29%). Predictive model performance measures, i.e., calibration and discrimination, were reported in 12% and 27% of studies, respectively. The majority of prediction studies in high impact journals do not follow current methodological recommendations, limiting their reliability and applicability. Please see later in the article for the Editors' Summary There are often times in our lives when we would like to be able to predict the future. Is the stock market going to go up, for example, or will it rain tomorrow? Being able predict future health is also important, both to patients and to physicians, and there is an increasing body of published clinical “prediction research.” Diagnostic prediction research investigates the ability of variables or test results to predict the presence or absence of a specific diagnosis. So, for example, one recent study compared the ability of two imaging techniques to diagnose pulmonary embolism (a blood clot in the lungs). Prognostic prediction research investigates the ability of various markers to predict future outcomes such as the risk of a heart attack. Both types of prediction research can investigate the predictive properties of patient characteristics, single variables, tests, or markers, or combinations of variables, tests, or markers (multivariable studies). Both types of prediction research can include also studies that build multivariable prediction models to guide patient management (model development), or that test the performance of models (validation), or that quantify the effect of using a prediction model on patient and physician behaviors and outcomes (impact assessment). With the increase in prediction research, there is an increased interest in the methodology of this type of research because poorly done or poorly reported prediction research is likely to have limited reliability and applicability and will, therefore, be of little use in patient management. In this systematic review, the researchers investigate the reporting and methods of prediction studies by examining the aims, design, participant selection, definition and measurement of outcomes and candidate predictors, statistical power and analyses, and performance measures included in multivariable prediction research articles published in 2008 in several general medical journals. In a systematic review, researchers identify all the studies undertaken on a given topic using a predefined set of criteria and systematically analyze the reported methods and results of these studies. The researchers identified all the multivariable prediction studies meeting their predefined criteria that were published in 2008 in six high impact general medical journals by browsing through all the issues of the journals (a hand search). They then scored the methods and reporting of each study using a comprehensive item list based on recent recommendations for the conduct of prediction research (for example, the reporting recommendations for tumor marker prognostic studies—the REMARK guidelines). Of 71 retrieved studies, 51 were predictor finding studies, 14 were prediction model development studies, three externally validated an existing model, and three reported on a model's impact on participant outcome. Study design, participant selection, definitions of outcomes and predictors, and predictor selection were generally well reported, but other methodological and reporting aspects of the studies were suboptimal. For example, despite many recommendations, continuous predictors were often dichotomized. That is, rather than using the measured value of a variable in a prediction model (for example, blood pressure in a cardiovascular disease prediction model), measurements were frequently assigned to two broad categories. Similarly, many of the studies failed to adequately estimate the sample size needed to minimize bias in predictor effects, and few of the model development papers quantified and validated the proposed model's predictive performance. These findings indicate that, in 2008, most of the prediction research published in high impact general medical journals failed to follow current guidelines for the conduct and reporting of clinical prediction studies. Because the studies examined here were published in high impact medical journals, they are likely to be representative of the higher quality studies published in 2008. However, reporting standards may have improved since 2008, and the conduct of prediction research may actually be better than this analysis suggests because the length restrictions that are often applied to journal articles may account for some of reporting omissions. Nevertheless, despite some encouraging findings, the researchers conclude that the poor reporting and poor methods they found in many published prediction studies is a cause for concern and is likely to limit the reliability and applicability of this type of clinical research. Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001221. The EQUATOR Network is an international initiative that seeks to improve the reliability and value of medical research literature by promoting transparent and accurate reporting of research studies; its website includes information on a wide range of reporting guidelines including the REMARK recommendations (in English and Spanish) A video of a presentation by Doug Altman, one of the researchers of this study, on improving the reporting standards of the medical evidence base, is available The Cochrane Prognosis Methods Group provides additional information on the methodology of prognostic research
登录
查看更多内容
影响因子：
9.3
作者：
Bossuyt, PM;Reitsma, JB;de Vet, HCW
通讯作者：
de Vet, HCW
影响因子：
7.2
作者：
Hayden, J. A.;Cote, P.;Bombardier, C.
通讯作者：
Bombardier, C.
影响因子：
7.2
作者：
Gorelick, Marc H.
通讯作者：
Gorelick, Marc H.
影响因子：
4
作者：
Biesheuvel, Cornelis J.;Vergouwe, Yvonne;Moons, Karel Gm
通讯作者：
Moons, Karel Gm
影响因子：
8.8
作者：
通讯作者：
--