Assessment of Variance Structure in ROC Studies

描述（由申请人提供）： 近几十年来，受试者工作特征（ROC）的范例，即真阳性率与假阳性率的实验室映射，在诊断技术的评估中发挥了核心作用，特别是在医学成像和计算机辅助诊断中。在最近的十年中，临床医生阅读图像的性能的巨大变化，以及随机效应ROC分析的出现，越来越多的认识，使多读者，多病例（MRMC）ROC范式的技术评估界占主导地位。这种范式的完全交叉版本，即所有读者在所有竞争模态中解释相同的案例，被许多人认为是对给定数量的真理验证案例统计上最强大的。这种能力是在面对高读取器和病例可变性时实现的，因为只有那些在竞争模态之间不相关的可变性分量才能掩盖确定模态之间差异的能力，并且完全交叉设计的目标之一是最小化这些分量。该领域的最新发展使我们能够估计MRMC-ROC类型实验中所有变异性来源和潜在相关性。这些信息提供了理解为什么某些设计比其他设计更成功的机会;它还允许人们根据较小的初步研究结果更好地设计大型关键试验。我们建议分析我们的大型数据库中的18个大型ROC型实验在这些方面。该项目的预期益处是深入了解设计方法，以实现更强大的统计方法来评估和比较竞争性诊断成像和计算机辅助模式，即，那些对收集患者病例和扩展放射科医师阅读时间的昂贵资源要求较低的系统。

DESCRIPTION (provided by applicant): In recent decades the paradigm of the receiver operating characteristic (ROC), namely, the laboratory mapping of the true-positive rate versus the false-positive rate, has played a central role in the assessment of diagnostic technologies, especially in medical imaging and computer-aided diagnosis. In the most recent decade, the growing awareness of the great variability in performance of clinicians reading images, together with the emergence of random-effects ROC analysis, have made the multiplereader, multiple-case (MRMC) ROC paradigm the dominant one in the technology assessment community. The fully crossed version of this paradigm, namely, where all readers interpret the same cases in all competing modalities, is considered by many as the most statistically powerful for a given number of truth-verified cases. This power is achieved in the face of high reader and case variability because only those components of variability that are uncorrelated across competing modalities mask the ability to determine a difference between the modalities, and one of the goals of the fully crossed design is to minimize these very components. Recent developments in the field give us the ability to estimate all of the sources of variability and the underlying correlations in MRMC-ROC type experiments. This information provides the opportunity to understand why some designs are more successful than others; it also allows one to better design a large pivotal trial from the results of a smaller pilot study. We propose to analyze our large database of 18 large ROC-type experiments in these terms. The expected benefit from this project will be the insight into design methods to achieve more statistically powerful approaches to the assessment and comparison of competing diagnostic imaging and computer-assist modalities, i.e., those that are the less demanding of the expensive resources of gathering patient cases and expanding radiologists reading time.