EVALUATION OF PERFORMANCE MEASURES FOR PREDICTIVE MODELS

预测模型的性能指标评估

• 批准号: 2653457
• 负责人: JOHN GRIFFITH
• 金额: $ 7.99万
• 依托单位: TUFTS MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 1997
• 资助国家: 美国
• 起止时间: 1997-09-30 至 1999-09-29
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/2653457
• 关键词: behavioral /social science research tag; clinical research; health science research; health science research analysis /evaluation; health services research tag; human data; mathematical model; outcomes research; statistics /biometry

Predictive mathematical models of dichotomous medical outcomes that generate an estimated probability that a patient will have an outcome are central to many clinical decision aids and are fundamental to comparative analyses of medical care based on risk- adjusted events. In such applications, inaccurate assessment of patient risk can have significant health care and health policy implications. Thus it is critical that researchers have methods for assessing the accuracy of the predictions from these models. Statistical methods for assessing the overall performance, calibration, and discrimination of these predictive models have been proposed and are currently being used. Unfortunately, the lack of a meaningful interpretation and thorough understanding by researchers of these measures of predictive performance limit their usefulness and the ability of researchers to evaluate and compare predictive models. Using data from several large controlled randomized trials, and previously developed simulation programs, this project will provide an evaluation of the appropriate use of these measures by addressing three questions: 1. What is the expected agreement between performance measures? 2. What is the sensitivity of each performance measure to model and data characteristics? 3. What is a meaningful interpretation of the calculated value for each statistic? these questions will be addressed in four phases. In phase one the mathematical formula used for each measure will be reformulated and compared. In phase two these performance measures will be applied and compared for existing predictive models. In phase three the performance measures will be applied to simulated data and predictive models. Based on the results of the prior phases, and input from clinical researchers, in phase four useful interpretations of the meaning and utility of each measure will be evaluated. This study should broaden the understanding of these measures of predictive performance for models of dichotomous outcomes, and improve the ability of health policy and health services researchers to properly assess the accuracy, and thus utility, of predictive models.

二分医疗结果的预测数学模型 其产生患者将具有 结果是许多临床决策辅助工具的核心， 基于风险的医疗护理比较分析的基础- 调整事件。 在这些应用中， 患者风险可能具有重要的医疗保健和健康政策 含义。 因此，研究人员必须有方法 评估这些模型预测的准确性。 评估整体性能、校准、 并提出了这些预测模型的判别方法 目前正在使用。 不幸的是，缺乏 研究者有意义的解释和透彻的理解 这些预测性能的测量限制了它们的有用性 以及研究人员评估和比较预测结果的能力 模型 利用几个大型随机对照试验的数据， 和以前开发的模拟程序，该项目将 评估这些措施的适当使用情况， 回答三个问题：1.预期的协议是什么 有什么区别吗 2. 每种的敏感度是多少 模型和数据特征的性能指标？ 3. 是什么 对每个统计量的计算值进行有意义的解释？ 这些问题将分四个阶段处理。 在第一阶段， 将重新制定用于每项措施的数学公式， 和比较 在第二阶段，这些绩效指标将 应用并比较现有的预测模型。 在第三阶段 性能指标将应用于模拟数据， 预测模型 根据前几个阶段的结果， 临床研究人员的投入，在第四阶段， 将评估每项措施的意义和效用。 这 研究应扩大对这些措施的理解， 二分结果模型的预测性能，以及 提高卫生政策和卫生服务研究人员的能力， 正确评估预测模型的准确性和实用性。