Human Observer Net: A Platform Tool for Human Observer Studies of Image Data
Human Observer Net: A Platform Tool for Human Observer Studies of Image Data
复制标题
DOI：
10.1148/radiol.211832
复制
发表时间：
2022-06-01
期刊：
影响因子：
19.7
通讯作者：
Jahnke，Paul
中科院分区：
文献类型：
--
作者：
Genske，Ulrich;Jahnke，Paul
当前用于图像人类观察者研究的软件应用在研究设计、平台独立性、多中心使用和评估方法方面缺乏灵活性，并且不是开源的，限制了可访问性和可扩展性。目的：开发一个用户友好的软件平台，使医学影像中高效的人体观察者研究具有研究设计的灵活性。材料与方法用于人体观察者成像研究的软件被设计为一个开源的web应用程序，以方便访问，平台无关的可用性和多中心研究。实现了用于研究创建、参与和结果管理的不同接口。该软件在2019年5月至2021年3月期间在人类观察者实验中进行了评估，其中跟踪了观察者反应的持续时间。14位放射科医生使用100分系统可用性量表评估和分级软件可用性。该应用程序在Chrome、Firefox、Safari和Edge浏览器上进行了测试。结果设计软件功能，支持视觉分级分析（VGA）、多选项强迫选择（m-AFC）、受试者工作特征（ROC）、定位ROC、自由反应ROC和定制设计。每幅图像或每组图像的平均反应时间分别为6.2秒±4.8秒（标准差）、5.8秒±4.7秒、8.7秒±5.7秒和6.0秒±4.5秒，分别为4张afc（160张图像）、4张afc（640张图像）、定位ROC和实验研究。系统可用性量表的平均得分为83±11（满分100分）。文档代码和应用程序的演示可在线获得（https://github.com/genskeu/HON, https://hondemo.pythonanywhere.com/）.ConclusionA)，用户友好且高效的开源应用程序是为人类读者实验开发的，可实现研究设计的通用性，以及独立于平台和多中心的可用性。©RSNA， 2022另见汤普森在本期的社论。
BackgroundCurrent software applications for human observer studies of images lack flexibility in study design, platform independence, multicenter use, and assessment methods and are not open source, limiting accessibility and expandability.PurposeTo develop a user-friendly software platform that enables efficient human observer studies in medical imaging with flexibility of study design.Materials and MethodsSoftware for human observer imaging studies was designed as an open-source web application to facilitate access, platform-independent usability, and multicenter studies. Different interfaces for study creation, participation, and management of results were implemented. The software was evaluated in human observer experiments between May 2019 and March 2021, in which duration of observer responses was tracked. Fourteen radiologists evaluated and graded software usability using the 100-point system usability scale. The application was tested in Chrome, Firefox, Safari, and Edge browsers.ResultsSoftware function was designed to allow visual grading analysis (VGA), multiple-alternative forced-choice (m-AFC), receiver operating characteristic (ROC), localization ROC, free-response ROC, and customized designs. The mean duration of reader responses per image or per image set was 6.2 seconds ± 4.8 (standard deviation), 5.8 seconds ± 4.7, 8.7 seconds ± 5.7, and 6.0 seconds ± 4.5 in four-AFC with 160 image quartets per reader, four-AFC with 640 image quartets per reader, localization ROC, and experimental studies, respectively. The mean system usability scale score was 83 ± 11 (out of 100). The documented code and a demonstration of the application are available online (https://github.com/genskeu/HON, https://hondemo.pythonanywhere.com/).ConclusionA user-friendly and efficient open-source application was developed for human reader experiments that enables study design versatility, as well as platform-independent and multicenter usability.© RSNA, 2022See also the editorial by Thompson in this issue.