EVALUATING THE PERFORMANCE OF RISK PREDICTION MODELS

由于预期寿命的增加，比以往任何时候都有更多的人生活在疾病和健康不良之中。医生通常需要估计患有特定疾病的患者的预后或评估他们发展疾病的可能性。预后研究提供了重要的证据，支持有关治疗，进一步测试或生活方式改变的临床决策。风险评分是医生预后信息的重要来源，结合患者和疾病特征来估计该个体特定结果的风险。然后，这些风险估计可用于指导患者和医生做出临床决策。风险评分越来越多地出现在医疗政策和临床实践指南中。开发风险评分是一个复杂的过程，涉及许多方法选择，决定了模型的价值？对患者的预测？结果。在制定风险评分后，必须证明其有效。患者不会从风险评分中获益，除非区分预后良好或不良的患者。通常对同一组患者进行性能评估，以得出导致乐观结果的风险评分。有必要评估风险评分吗？在一组新的患者身上进行的研究，称为验证研究。目前还没有就进行这种验证研究的最佳方式达成共识。我们计划调查影响风险评分表现的验证研究的几个方法学方面。这将使我们能够为未来的验证研究制定指导。我们将设法确定需要研究多少个体才能对风险评分进行可靠的评估，并就处理缺失数据的最佳方法提供指导。另一个问题是处理连续测量的方式。通常，这些测量值只是被分组为高值或低值，这种方法意味着两个特定的患者，其值刚好高于和刚好低于所选择的临界点，令人难以置信地被表征为具有非常不同的血压。我们将使用数百万份全科患者记录的数据集来解决所有这些问题，使用医学文献中公布的风险评分以及互联网上的公共领域。我们的研究结果将为研究人员设计和进行高质量的研究提供指导，以评估绩效风险评分。这将提高医生和决策者可获得的预后信息的质量。

More people are living with disease and ill-health than ever before due to increased life expectancy. Doctors often need to estimate the prognosis of a patient with a specific disease or assess their likelihood of developing a disease. Prognosis research provides crucial evidence underpinning clinical decisions about treatment, further tests, or lifestyle changes. Risk scores are a valuable source of prognostic information for doctors, combining patient and disease characteristics to estimate the risk of a specific outcome for that individual. These estimates of risk can then be used to guide patients and doctors in making clinical decisions. Risk scores feature increasingly in healthcare policy and clinical practice guidelines. Developing risk scores is a complex process involving a number of methodological choices that dictate the value of the model?s predictions on patients? outcomes. After developing a risk score, it is essential to demonstrate that it works. Patients will not benefit from a risk score unless distinguishes between those with a good or poor prognosis. Performance is often assessed on the same group of patients used to derive the risk score leading to optimistic results. It is necessary to evaluate the risk score?s performance on a new group of patients, called a validation study. There is currently no consensus on the best way to carry out such a validation study. We plan to investigate several methodological aspects of validation studies that influence the performance of a risk score. This will enable us to develop guidance for the conduct of future validation studies. We shall seek to ascertain how many individuals to study to allow reliable evaluation of a risk score, and provide guidance on the best way to deal with missing data. Another issue is the manner in which continuous measurements are handled. Often values of such measurements are just grouped as high or low values, an approach that implies that two particular patients with values just above and just below the chosen cut-point are implausibly characterised as having very different prognoses. We will use a data set of several million general practice patient records to address all these issues, using risk scores published in the medical literature and that are available in the public domain on the internet. Our results will provide researchers with guidance on designing and conducting high quality studies that evaluate the performance risk scores. This will improve the quality of prognostic information available to doctors and policy-makers.