Comparing Performance Under FROC Paradigm - A Non Parametirc Approach

描述（由申请人提供）：常规进行观察者（和非观察者）性能研究，以评估医学成像技术和实践，以便在临床环境中以及在监管审批过程中使用。迄今为止，这些评估的很大一部分，包括但不限于在医学成像领域进行的评估，需要任务特定的ROC类型反应（评分）进行分析。最近的进步导致了分析工具的发展，这些工具可以比较在FROC范式下确定的数据，对于该领域的许多应用来说，FROC范式可能比ROC范式具有实质性的优势。FROC方法也可能被证明更密切地遵循（模仿）实际的临床诊断过程，因此，基于研究结果的概括性和推断可能比其他方法更有效（相关性）。该项目的总体目标是提高我们对系统性能研究中遇到的实际（方法学）和计算（分析）相关问题的理解，当实验范式包括FROC类型评级和观察者，计算机化方案（例如CAD）或两者的组合成为诊断系统的组成部分时。在分析ROC类型研究中确定的评分数据的新方法的基础上，我们建议开发基于非参数排列的测试来比较配对设计中的FROC评分数据集。基本方法比较所有对实际阴性和阳性结果之间的观察结果（例如，特定位置的癌症），以使用非参数方法构建性能的总结指数。将开发和测试无条件和条件（适当时）测试。我们将使用广泛的模拟和实验确定的数据集，对所提出的方法进行全面的研究，包括其在不同研究条件下的优点和局限性。

DESCRIPTION (provided by applicant): Observer (and non-observer) performance studies are routinely performed to evaluate technologies and practices in medical imaging for possible use in the clinical environment as well as during the regulatory approval process. A large fraction of these evaluations to date, including but not limited to those performed in the field of medical imaging, required task specific ROC type response (ratings) for the analyses. Recent advancements have led to the development of analytical tools that enable the comparison of data ascertained under the FROC paradigm, which may have substantial advantages over the ROC paradigm for many of the applications of interest in this field. The FROC approach may also prove to more closely follow (emulate) the actual clinical diagnostic process, hence generalizability of and inferences based on study results could have more validity (relevance) than that attainable with other approaches, the overall objective of this project is to improve our understanding of the practical (methodological) and computational (analyses) related issues that one encounters during systems' performance studies when the experimental paradigm includes FROC type ratings and the observer, a computerized scheme (e.g. CAD), or a combination of both become an integral part of the diagnostic system. Building on a new approach to analyze scoring data ascertained during the performance of ROC type studies, we propose to develop non- parametric permutation based tests to compare FROC rating datasets in a paired design. The underlying approach compares observations between all pairs of actually negative and positive findings (e.g. cancer at a specific location) to construct summary indices of performance using a non-parametric methodology. Both unconditional and conditional (when appropriate) tests will be developed and tested. The proposed, including its advantages and limitations under different study conditions, will be investigated in a comprehensive manner using both extensive simulations as experimentally ascertained datasets.