EVALUATION OF PERFORMANCE MEASURES FOR PREDICTIVE MODELS

二分法医疗结果的预测数学模型 这产生了一个估计的概率，病人将会有一个 结果是许多临床决策辅助工具的核心，并且 基于风险的医疗保健比较分析的基础-- 调整后的事件。在这类应用中，不准确的评估 患者风险可以有重要的医疗保健和健康政策 这意味着什么。因此，至关重要的是，研究人员必须有方法来 评估这些模型预测的准确性。 评估总体性能的统计方法、校准、 并提出了对这些预测模型的判别方法 目前正在使用中。不幸的是，缺乏一个 研究者的有意义的解释和透彻的理解 这些预测绩效的衡量标准限制了它们的有用性 以及研究人员评估和比较预测的能力 模特们。使用来自几个大型随机对照试验的数据， 和以前开发的模拟程序，这个项目将 通过以下方式对这些措施的适当使用进行评估 回答三个问题：1.预期的协议是什么 在两个绩效指标之间？2.每个指标的敏感度是什么 模型和数据特征的性能测量？3.什么是 对每个统计数据的计算值进行有意义的解释？ 这些问题将分四个阶段加以解决。在第一阶段， 用于每种测量的数学公式将被重新表述 并进行了比较。在第二阶段，这些绩效衡量标准将是 对现有的预测模型进行了应用和比较。在第三阶段 性能指标将应用于模拟数据和 预测性模型。基于先前阶段的结果，以及 来自临床研究人员的意见，在第四阶段对 将对每项措施的意义和效用进行评估。这 研究应扩大对这些措施的理解 二分结果模型的预测性能，以及 提高卫生政策和卫生服务研究人员的能力 正确评估预测模型的准确性，从而评估其实用性。

Predictive mathematical models of dichotomous medical outcomes that generate an estimated probability that a patient will have an outcome are central to many clinical decision aids and are fundamental to comparative analyses of medical care based on risk- adjusted events. In such applications, inaccurate assessment of patient risk can have significant health care and health policy implications. Thus it is critical that researchers have methods for assessing the accuracy of the predictions from these models. Statistical methods for assessing the overall performance, calibration, and discrimination of these predictive models have been proposed and are currently being used. Unfortunately, the lack of a meaningful interpretation and thorough understanding by researchers of these measures of predictive performance limit their usefulness and the ability of researchers to evaluate and compare predictive models. Using data from several large controlled randomized trials, and previously developed simulation programs, this project will provide an evaluation of the appropriate use of these measures by addressing three questions: 1. What is the expected agreement between performance measures? 2. What is the sensitivity of each performance measure to model and data characteristics? 3. What is a meaningful interpretation of the calculated value for each statistic? these questions will be addressed in four phases. In phase one the mathematical formula used for each measure will be reformulated and compared. In phase two these performance measures will be applied and compared for existing predictive models. In phase three the performance measures will be applied to simulated data and predictive models. Based on the results of the prior phases, and input from clinical researchers, in phase four useful interpretations of the meaning and utility of each measure will be evaluated. This study should broaden the understanding of these measures of predictive performance for models of dichotomous outcomes, and improve the ability of health policy and health services researchers to properly assess the accuracy, and thus utility, of predictive models.