Verification of modified receiver-operating characteristic software using simulated rating data
Verification of modified receiver-operating characteristic software using simulated rating data
复制标题
DOI：
10.1007/s12194-018-0479-9
复制
发表时间：
2018-12-01
影响因子：
1.6
通讯作者：
Hara, Takeshi
中科院分区：
文献类型：
--
作者：
Shiraishi, Junji;Fukuoka, Daisuke;Hara, Takeshi
ROCKIT是由Metz等人开发的接收机工作特性（ROC）曲线拟合软件包。在20世纪90年代早期，它是世界上非常频繁使用的ROC软件。除ROCKIT外，还开发了DBM-MRMC软件，用于多阅读器多病例ROC曲线下平均面积（auc）差异分析。由于这款旧软件无法在安装Windows 7或更新的操作系统的个人电脑上运行，我们开发了一款新软件，采用了相同的基本算法，只是做了一些小小的修改。在本研究中，我们验证了修改后的软件，并使用模拟评级数据测试了诊断准确性指标之间的差异。在我们的模拟模型中，所有数据都是使用目标AUC和一个二正态参数b生成的。在使用模拟评级数据进行ROC曲线拟合时，我们改变了四个因素：病例样本总数、阳性与阴性病例的比例、一个二正态参数b和预设的AUC。为了研究从我们的软件获得的统计测试结果与现有软件之间的差异，我们生成了模拟评级数据集，其中包含三个级别的病例难度和从两种模式获得的auc的三个级别差异。仿真结果表明，新软件和现有软件估计的auc高度相关（R>0.98），统计检验结果一致性较高（85%以上）。总之，我们相信我们修改后的软件和现有的软件一样有能力。
ROCKIT, which is a receiver-operating characteristic (ROC) curve-fitting software package, was developed by Metz et al. In the early 1990s, it is a very frequently used ROC software throughout the world. In addition to ROCKIT, DBM-MRMC software was developed for multi-reader multi-case analysis of the difference in average area under ROC curves (AUCs). Because this old software cannot run on a PC with Windows 7 or a more recent operating system, we developed new software that employs the same basic algorithms with minor modifications. In this study, we verified our modified software and tested the differences between the index of diagnostic accuracies using simulated rating data. In our simulation model, all data were generated using target AUCs and a binormal parameter b. In ROC curve fitting with simulated rating data, we varied four factors: the total number of case samples, the ratio of positive-to-negative cases, a binormal parameter b, and the preset AUC. To investigate the differences between the statistical test results obtained from our software and the existing software, we generated simulated rating data sets with three levels of case difficulty and three degrees of difference in AUCs obtained from two modalities. As a result of the simulation, the AUCs estimated by the new and existing software were highly correlated (R>0.98), and there were high agreements (85% or more) in the statistical test results. In conclusion, we believe that our modified software is as capable as the existing software.