Assessment of Variance Structure in ROC Studies

描述(由申请人提供)： 近几十年来，受试者工作特征(ROC)的范例，即真阳性率与假阳性率的实验室图谱，在诊断技术的评估中发挥了核心作用，特别是在医学成像和计算机辅助诊断中。在最近的十年中，临床医生对图像阅读能力的巨大差异的认识日益增强，以及随机效应ROC分析的出现，使得多读取器，多病例(MRMC)ROC范式成为技术评估界的主导范式。这一范式的完全交叉版本，即所有读者在所有相互竞争的模式中解释相同的案例，被许多人认为是对给定数量的真实核实案例最具统计效力的。这种能力是在面对高度的读者和病例可变性的情况下实现的，因为只有那些在竞争的医疗模式中不相关的可变性组件才能掩盖确定医疗模式之间差异的能力，而完全交叉设计的目标之一就是将这些组件降至最低。该领域的最新发展使我们能够估计MRMC-ROC类型实验中所有可变性的来源和潜在的相关性。这些信息提供了了解为什么某些设计比其他设计更成功的机会；它还允许人们根据较小的先导性研究的结果更好地设计大型关键试验。我们建议用这些术语来分析我们的18个大型ROC类型实验的大型数据库。该项目的预期收益将是对设计方法的洞察，以实现更强大的统计方法来评估和比较相互竞争的诊断成像和计算机辅助模式，即那些对收集患者病例和延长放射科医生阅读时间的昂贵资源要求较低的方法。

DESCRIPTION (provided by applicant): In recent decades the paradigm of the receiver operating characteristic (ROC), namely, the laboratory mapping of the true-positive rate versus the false-positive rate, has played a central role in the assessment of diagnostic technologies, especially in medical imaging and computer-aided diagnosis. In the most recent decade, the growing awareness of the great variability in performance of clinicians reading images, together with the emergence of random-effects ROC analysis, have made the multiplereader, multiple-case (MRMC) ROC paradigm the dominant one in the technology assessment community. The fully crossed version of this paradigm, namely, where all readers interpret the same cases in all competing modalities, is considered by many as the most statistically powerful for a given number of truth-verified cases. This power is achieved in the face of high reader and case variability because only those components of variability that are uncorrelated across competing modalities mask the ability to determine a difference between the modalities, and one of the goals of the fully crossed design is to minimize these very components. Recent developments in the field give us the ability to estimate all of the sources of variability and the underlying correlations in MRMC-ROC type experiments. This information provides the opportunity to understand why some designs are more successful than others; it also allows one to better design a large pivotal trial from the results of a smaller pilot study. We propose to analyze our large database of 18 large ROC-type experiments in these terms. The expected benefit from this project will be the insight into design methods to achieve more statistically powerful approaches to the assessment and comparison of competing diagnostic imaging and computer-assist modalities, i.e., those that are the less demanding of the expensive resources of gathering patient cases and expanding radiologists reading time.