EVALUATING THE PERFORMANCE OF RISK PREDICTION MODELS

由于预期寿命的延长，比以往任何时候都有更多的人患有疾病和健康不佳。医生经常需要评估患有特定疾病的患者的预后，或评估他们患上某种疾病的可能性。预后研究为有关治疗、进一步检查或改变生活方式的临床决策提供了至关重要的证据。对于医生来说，风险评分是一个有价值的预后信息来源，它结合了患者和疾病的特征，以估计该个人特定结果的风险。然后，这些风险评估可以用来指导患者和医生做出临床决策。风险评分越来越多地出现在医疗保健政策和临床实践指南中。制定风险评分是一个复杂的过程，涉及许多方法选择，这些方法决定了模型的价值？S对患者的预测？结果。在制定了风险评分之后，必须证明它是有效的。除非区分预后好或差的患者，否则患者不会从风险评分中受益。通常对用于得出风险评分的同一组患者进行绩效评估，从而得出乐观的结果。有必要评估风险评分？S对一组新患者的表现，称为验证性研究。目前还没有就进行这种验证性研究的最佳方式达成共识。我们计划调查验证性研究的几个方法学方面，它们影响风险评分的表现。这将使我们能够为未来的验证研究制定指导方针。我们将设法确定需要研究多少人才能对风险得分进行可靠的评估，并就处理丢失数据的最佳方法提供指导。另一个问题是处理连续测量的方式。通常，这些测量值只是分为高值或低值，这一方法意味着，两个值略高于和略低于所选临界点的特定患者被描述为预后截然不同，这令人难以置信。我们将使用数百万全科医生患者记录的数据集来解决所有这些问题，使用在医学文献中发布的风险评分，这些评分可以在互联网上的公共领域获得。我们的结果将为研究人员设计和进行高质量的研究提供指导，以评估性能风险分数。这将提高医生和政策制定者可获得的预后信息的质量。

More people are living with disease and ill-health than ever before due to increased life expectancy. Doctors often need to estimate the prognosis of a patient with a specific disease or assess their likelihood of developing a disease. Prognosis research provides crucial evidence underpinning clinical decisions about treatment, further tests, or lifestyle changes. Risk scores are a valuable source of prognostic information for doctors, combining patient and disease characteristics to estimate the risk of a specific outcome for that individual. These estimates of risk can then be used to guide patients and doctors in making clinical decisions. Risk scores feature increasingly in healthcare policy and clinical practice guidelines. Developing risk scores is a complex process involving a number of methodological choices that dictate the value of the model?s predictions on patients? outcomes. After developing a risk score, it is essential to demonstrate that it works. Patients will not benefit from a risk score unless distinguishes between those with a good or poor prognosis. Performance is often assessed on the same group of patients used to derive the risk score leading to optimistic results. It is necessary to evaluate the risk score?s performance on a new group of patients, called a validation study. There is currently no consensus on the best way to carry out such a validation study. We plan to investigate several methodological aspects of validation studies that influence the performance of a risk score. This will enable us to develop guidance for the conduct of future validation studies. We shall seek to ascertain how many individuals to study to allow reliable evaluation of a risk score, and provide guidance on the best way to deal with missing data. Another issue is the manner in which continuous measurements are handled. Often values of such measurements are just grouped as high or low values, an approach that implies that two particular patients with values just above and just below the chosen cut-point are implausibly characterised as having very different prognoses. We will use a data set of several million general practice patient records to address all these issues, using risk scores published in the medical literature and that are available in the public domain on the internet. Our results will provide researchers with guidance on designing and conducting high quality studies that evaluate the performance risk scores. This will improve the quality of prognostic information available to doctors and policy-makers.