Improved DBM ROC Methods for Diagnostic Radiology

改进的诊断放射学 DBM ROC 方法

• 批准号: 7503120
• 负责人: Kevin Staley Berbaum
• 金额: $ 7.3万
• 依托单位: UNIVERSITY OF IOWA
• 依托单位国家: 美国
• 财政年份: 2003
• 资助国家: 美国
• 起止时间: 2003-04-01 至 2011-03-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7503120
• 关键词: Accounting; Address; Adoption; Affect; Analysis of Variance; Architecture; Caring; Computer software; Computer-Assisted Diagnosis; Cost-Benefit Analysis; Costs and Benefits; Data; Data Set; Development; Diagnostic; Diagnostic radiologic examination; Disease; Disease regression; Effectiveness; Equilibrium; Evaluation Research; Experimental Designs; Freedom; Goals; Health Care Costs; Health Policy; Image; Lead; Least-Squares Analysis; Lesion; Location; Measures; Methodology; Methods; Modality; Modeling; Neuro-Oncological Ventral Antigen 2; Numbers; Outcome; Patients; Performance; Physicians; Population; Prevalence; Procedures; Radiology Specialty; Range; Reader; Reading; Receiver Operating Characteristics; Relative (related person); Research; Research Personnel; SAS; Sampling; Sensitivity and Specificity; Societies; Specific qualifier value; Standards of Weights and Measures; Structure; System; Techniques; Variant; base; clinically relevant; coping; cost; design; diagnostic accuracy; improved; indexing; innovation; interest; novel; open source; prevent; programs; research study; size; software development; statistics; user-friendly

This proposal addresses a fundamental methodological problem in diagnostic radiology research: evaluation of diagnostic performance of imaging modalities. Receiver operating characteristic (ROC) analysis is an accepted procedure for measuring how well imaging modalities allow physicians to detect disease. Dorfman, Berbaum and Metz (1992) (DBM) developed a statistical methodology to analyze ROC studies with multiple readers, taking into account both reader-sample and case-sample variation. DBM methodology is the most widely used method to assess diagnostic performance of new imaging systems. Unfortunately, the usefulness and effectivenessof DBM method is limited: DBM only handles a narrow range of experimental designs and cannot cope with missing data. The soundness of DBM conclusions may be in question when there are many more or many less diseased patients than non-diseased patients. DBM does not fit ROC curves using ordinal regression, restraining further development of ROC models and preventing the incorporation of more advanced statistical techniques. DBM cannot use patient information to improve statistical power. DBM only compares diagnostic systems on accuracy of diagnosis, not on the benefits and costs for patients and society. Our goal is to remove these limitations by extending DBM method's range of application and by improving its effectiveness. The specific aims are:(1) formulate a more general model for DBM MRMC analysis using the estimated generalized least squares method to cope with computer-aided diagnosis experiments, multiple covariate effects, missing data, designs split on readers or cases, and readers reading different cases; (2) develop an option that determines the sensitivity of inferencesto the variability of the variance-covariance estimates when the number of cases is small, and when the number of cases is moderate but the proportion of normal to abnormal cases is close to one or zero; (3) provide a validated and easy-to-use ordinal regression option for ROC curve fitting within our major DBM MRMC method that offers familiar and interpretable ROC-style results; (4) extend DBM MRMC to include a disease-specific patient-level covariate to discover how the covariate affects accuracy and thereby gain more statistical power by explaining variation in outcomes; and (5) develop DBM MRMC methodology for cost/benefit analysis to focus the comparison of diagnostic systems on clinically relevant parts of ROC curves and provide a quantitative basis for health policy decisions. Completing these aims will enhance the effectiveness of the primary method used to assess the diagnostic performance of new imaging systems. This will lead to better radiology research, which will raise the quality and lower the cost of health care.

该提案解决了诊断放射学研究中的一个基本方法学问题： 评价成像模式的诊断性能。受试者工作特性 分析是一个公认的程序，用于衡量成像模式如何让医生检测到 疾病Dorfman，Berbaum和梅斯（1992）（DBM）开发了一种统计方法来分析ROC 多个读者的研究，同时考虑读者样本和案例样本的变化。DBM 方法是最广泛使用的方法来评估新的成像系统的诊断性能。 不幸的是，DBM方法的有用性和有效性是有限的：DBM只处理狭窄的 实验设计的范围，不能科普缺失的数据。DBM结论的合理性 当患病患者比未患病患者多或少时，可能存在问题。 DBM不能使用有序回归拟合ROC曲线，限制了ROC模型的进一步发展， 这就阻碍了更先进的统计技术的应用。DBM无法使用患者信息 以提高统计能力。DBM仅在诊断准确性方面比较诊断系统，而不是在 对患者和社会的好处和成本。我们的目标是通过扩展DBM来消除这些限制 方法的应用范围和提高其有效性。 具体目标是：（1）利用估计的 广义最小二乘法，以科普计算机辅助诊断实验，多协变量 影响，缺失数据，设计分裂读者或情况下，和读者阅读不同的情况下;（2）开发 确定推断对方差-协方差估计值的变异性的敏感性的选项 当病例数较少时，以及当病例数适中但 正常到异常情况接近1或0;（3）提供一个有效且易于使用的顺序 在我们的主要DBM MRMC方法中，为ROC曲线拟合提供了回归选项， 可解释的ROC型结果;（4）扩展DBM MRMC以包括疾病特异性患者水平协变量 发现协变量如何影响准确性，从而通过解释 结果的变化;以及（5）开发DBM MRMC成本/效益分析方法， 比较诊断系统的ROC曲线的临床相关部分，并提供定量的 健康决策的基础。完成这些目标将提高小学的有效性， 用于评估新成像系统诊断性能的方法。这将导致更好的 放射学研究，这将提高医疗质量，降低医疗成本。