Improved DBM ROC Methods for Diagnostic Radiology

描述（由申请人提供）：本提案解决了诊断放射学研究中的一个基本方法学问题：评估成像模式的诊断性能。接受者工作特征（ROC）分析是一种公认的程序，用于测量成像模式允许医生检测疾病的程度。Dorfman， Berbaum和Metz (1992) （DBM）开发了一种统计方法来分析具有多个读者的ROC研究，同时考虑了读者-样本和病例-样本差异。DBM方法是评估新成像系统诊断性能最广泛使用的方法。不幸的是，DBM方法的有用性和有效性是有限的：DBM只能处理一个狭窄的实验设计范围，不能处理缺失的数据。当患病患者比非患病患者多或少时，DBM结论的可靠性可能会受到质疑。DBM不能使用有序回归拟合ROC曲线，限制了ROC模型的进一步发展，也阻碍了更先进的统计技术的引入。DBM不能使用患者信息来提高统计能力。DBM只比较诊断系统的诊断准确性，而不比较患者和社会的收益和成本。我们的目标是通过扩展DBM方法的应用范围和提高其有效性来消除这些限制。具体目标是：(1)针对计算机辅助诊断实验、多协变量效应、数据缺失、设计对读者或病例的分割、读者阅读不同病例等情况，采用估计广义最小二乘法建立更通用的DBM MRMC分析模型；(2)开发一种选项，确定当病例数量较少时，以及当病例数量适中但正常病例与异常病例的比例接近于1或0时，推断对方差-协方差估计变异性的敏感性；(3)在我们的主要DBM MRMC方法中，为ROC曲线拟合提供了一个经过验证且易于使用的有序回归选项，该选项提供了熟悉且可解释的ROC风格结果；(4)将DBM MRMC扩展到包含特定疾病的患者水平的协变量，以发现协变量如何影响准确性，从而通过解释结果的变化获得更多的统计能力；(5)发展用于成本/收益分析的DBM MRMC方法，将诊断系统的比较重点放在ROC曲线的临床相关部分上，并为卫生政策决策提供定量依据。完成这些目标将提高用于评估新成像系统诊断性能的主要方法的有效性。这将导致更好的放射学研究，从而提高医疗质量，降低医疗成本。

DESCRIPTION (provided by applicant): This proposal addresses a fundamental methodological problem in diagnostic radiology research: evaluation of diagnostic performance of imaging modalities. Receiver operating characteristic (ROC) analysis is an accepted procedure for measuring how well imaging modalities allow physicians to detect disease. Dorfman, Berbaum and Metz (1992) (DBM) developed a statistical methodology to analyze ROC studies with multiple readers, taking into account both reader-sample and case-sample variation. DBM methodology is the most widely used method to assess diagnostic performance of new imaging systems. Unfortunately, the usefulness and effectiveness of DBM method is limited: DBM only handles a narrow range of experimental designs and cannot cope with missing data. The soundness of DBM conclusions may be in question when there are many more or many less diseased patients than non-diseased patients. DBM does not fit ROC curves using ordinal regression, restraining further development of ROC models and preventing the incorporation of more advanced statistical techniques. DBM cannot use patient information to improve statistical power. DBM only compares diagnostic systems on accuracy of diagnosis, not on the benefits and costs for patients and society. Our goal is to remove these limitations by extending DBM method's range of application and by improving its effectiveness. The specific aims are: (1) formulate a more general model for DBM MRMC analysis using the estimated generalized least squares method to cope with computer-aided diagnosis experiments, multiple covariate effects, missing data, designs split on readers or cases, and readers reading different cases; (2) develop an option that determines the sensitivity of inferences to the variability of the variance-covariance estimates when the number of cases is small, and when the number of cases is moderate but the proportion of normal to abnormal cases is close to one or zero; (3) provide a validated and easy-to-use ordinal regression option for ROC curve fitting within our major DBM MRMC method that offers familiar and interpretable ROC-style results; (4) extend DBM MRMC to include a disease-specific patient-level covariate to discover how the covariate affects accuracy and thereby gain more statistical power by explaining variation in outcomes; and (5) develop DBM MRMC methodology for cost/benefit analysis to focus the comparison of diagnostic systems on clinically relevant parts of ROC curves and provide a quantitative basis for health policy decisions. Completing these aims will enhance the effectiveness of the primary method used to assess the diagnostic performance of new imaging systems. This will lead to better radiology research, which will raise the quality and lower the cost of health care.