Area under the Free-Response ROC Curve (FROC) and a Related Summary Index
Area under the Free-Response ROC Curve (FROC) and a Related Summary Index
复制标题
DOI：
10.1111/j.1541-0420.2008.01049.x
复制
发表时间：
2009-03-01
期刊：
影响因子：
1.9
通讯作者：
Gur, David
中科院分区：
文献类型：
--
作者：
Bandos, Andriy I.;Rockette, Howard E.;Gur, David
诊断系统的自由反应评估在与受试者内的一个或多个“异常”的检测、定位和分类相关的领域中继续获得认可。自由响应受试者工作特性（FROC）曲线是一种用于同时表征自由响应系统在所有决策阈值下的性能的工具。虽然在ROC分析中很好地认识到在所有决策阈值上总结整个曲线的单个指数的重要性（例如，例如，在一个实施例中，ROC曲线下的面积），目前还没有一个被广泛接受的总结下FROC范例评估的系统。在这篇文章中，我们提出了一个新的指标的自由响应性能在所有的决策阈值同时，并开发了一个非参数的方法进行分析。从代数上讲，所提出的汇总指数是经验FROC曲线下的面积，该面积因错误标记的数量而受到惩罚，因检测到的异常的分数而受到奖励，并因目标大小（或“接受半径”）的影响而进行调整。几何上，建议的指数可以被解释为一个人工的“猜测”的自由反应过程的平均性能优越性的措施，它代表了一个类比的ROC曲线和“猜测”或对角线之间的区域。我们推导出理想的自助估计的方差，它可以用于重采样的渐近自助置信区间的建设和使用标准表达式的样本量估计。建议的程序是免费的任何参数假设，并不需要假设的独立性的观察对象。我们提供了一个从诊断成像研究中采样的数据集的例子，并进行模拟，证明所考虑的样本量和参数范围的适当性的开发程序。
Free-response assessment of diagnostic systems continues to gain acceptance in areas related to the detection, localization, and classification of one or more "abnormalities" within a subject. A free-response receiver operating characteristic (FROC) curve is a tool for characterizing the performance of a free-response system at all decision thresholds simultaneously. Although the importance of a single index summarizing the entire curve over all decision thresholds is well recognized in ROC analysis (e. g., area under the ROC curve), currently there is no widely accepted summary of a system being evaluated under the FROC paradigm. In this article, we propose a new index of the free-response performance at all decision thresholds simultaneously, and develop a nonparametric method for its analysis. Algebraically, the proposed summary index is the area under the empirical FROC curve penalized for the number of erroneous marks, rewarded for the fraction of detected abnormalities, and adjusted for the effect of the target size (or "acceptance radius"). Geometrically, the proposed index can be interpreted as a measure of average performance superiority over an artificial "guessing" free-response process and it represents an analogy to the area between the ROC curve and the "guessing" or diagonal line. We derive the ideal bootstrap estimator of the variance, which can be used for a resampling-free construction of asymptotic bootstrap confidence intervals and for sample size estimation using standard expressions. The proposed procedure is free from any parametric assumptions and does not require an assumption of independence of observations within a subject. We provide an example with a dataset sampled from a diagnostic imaging study and conduct simulations that demonstrate the appropriateness of the developed procedure for the considered sample sizes and ranges of parameters.