Comparing Performance Under FROC Paradigm - A Non Parametirc Approach

描述（由申请人提供）：常规进行观察员（和非观察员）性能研究，以评价可能用于临床环境以及监管批准过程中的医学成像技术和实践。迄今为止，这些评估中的很大一部分，包括但不限于在医学成像领域进行的评估，需要任务特定的ROC类型响应（评级）进行分析。最近的进步导致了分析工具的发展，使FROC范式下确定的数据进行比较，这可能有很大的优势，在这一领域的许多感兴趣的应用程序的ROC范式。FROC方法也可能被证明更紧密地遵循（模拟）实际的临床诊断过程，因此基于研究结果的概括性和推断可能具有更高的有效性（相关性）比其他方法可达到的程度，该项目的总体目标是提高我们对实际问题的理解，（方法论）和计算（分析）当实验范式包括FROC类型评级和观察者时，在系统性能研究中遇到的相关问题，计算机化方案（例如CAD）或两者的组合成为诊断系统的组成部分。基于一种新的方法来分析在ROC类型研究的性能期间确定的评分数据，我们建议开发基于非参数排列的检验来比较配对设计中的FROC评级数据集。基本方法比较所有实际阴性和阳性结果对（例如特定位置的癌症）之间的观察结果，以使用非参数方法构建性能的汇总指数。将开发和测试无条件和有条件（适当时）测试。建议，包括其优点和局限性，在不同的研究条件下，将在一个全面的方式进行调查，使用两个广泛的模拟实验确定的数据集。

DESCRIPTION (provided by applicant): Observer (and non-observer) performance studies are routinely performed to evaluate technologies and practices in medical imaging for possible use in the clinical environment as well as during the regulatory approval process. A large fraction of these evaluations to date, including but not limited to those performed in the field of medical imaging, required task specific ROC type response (ratings) for the analyses. Recent advancements have led to the development of analytical tools that enable the comparison of data ascertained under the FROC paradigm, which may have substantial advantages over the ROC paradigm for many of the applications of interest in this field. The FROC approach may also prove to more closely follow (emulate) the actual clinical diagnostic process, hence generalizability of and inferences based on study results could have more validity (relevance) than that attainable with other approaches, the overall objective of this project is to improve our understanding of the practical (methodological) and computational (analyses) related issues that one encounters during systems' performance studies when the experimental paradigm includes FROC type ratings and the observer, a computerized scheme (e.g. CAD), or a combination of both become an integral part of the diagnostic system. Building on a new approach to analyze scoring data ascertained during the performance of ROC type studies, we propose to develop non- parametric permutation based tests to compare FROC rating datasets in a paired design. The underlying approach compares observations between all pairs of actually negative and positive findings (e.g. cancer at a specific location) to construct summary indices of performance using a non-parametric methodology. Both unconditional and conditional (when appropriate) tests will be developed and tested. The proposed, including its advantages and limitations under different study conditions, will be investigated in a comprehensive manner using both extensive simulations as experimentally ascertained datasets.