EVALUATING THE PERFORMANCE OF RISK PREDICTION MODELS

点击翻译按钮获取中文摘要

More people are living with disease and ill-health than ever before due to increased life expectancy. Doctors often need to estimate the prognosis of a patient with a specific disease or assess their likelihood of developing a disease. Prognosis research provides crucial evidence underpinning clinical decisions about treatment, further tests, or lifestyle changes. Risk scores are a valuable source of prognostic information for doctors, combining patient and disease characteristics to estimate the risk of a specific outcome for that individual. These estimates of risk can then be used to guide patients and doctors in making clinical decisions. Risk scores feature increasingly in healthcare policy and clinical practice guidelines. Developing risk scores is a complex process involving a number of methodological choices that dictate the value of the model?s predictions on patients? outcomes. After developing a risk score, it is essential to demonstrate that it works. Patients will not benefit from a risk score unless distinguishes between those with a good or poor prognosis. Performance is often assessed on the same group of patients used to derive the risk score leading to optimistic results. It is necessary to evaluate the risk score?s performance on a new group of patients, called a validation study. There is currently no consensus on the best way to carry out such a validation study. We plan to investigate several methodological aspects of validation studies that influence the performance of a risk score. This will enable us to develop guidance for the conduct of future validation studies. We shall seek to ascertain how many individuals to study to allow reliable evaluation of a risk score, and provide guidance on the best way to deal with missing data. Another issue is the manner in which continuous measurements are handled. Often values of such measurements are just grouped as high or low values, an approach that implies that two particular patients with values just above and just below the chosen cut-point are implausibly characterised as having very different prognoses. We will use a data set of several million general practice patient records to address all these issues, using risk scores published in the medical literature and that are available in the public domain on the internet. Our results will provide researchers with guidance on designing and conducting high quality studies that evaluate the performance risk scores. This will improve the quality of prognostic information available to doctors and policy-makers.