Utilizing a Digital Swarm Intelligence Platform to Improve Consensus Among Radiologists and Exploring Its Applications.
Utilizing a Digital Swarm Intelligence Platform to Improve Consensus Among Radiologists and Exploring Its Applications.
复制标题
DOI：
10.1007/s10278-022-00662-3
复制
发表时间：
2023-04
影响因子：
4.4
通讯作者：
Majumdar, Sharmila
中科院分区：
文献类型：
--
作者：
Shah, Rutwik;Nunes, Bruno Astuto Arouche;Gleason, Tyler;Fletcher, Will;Banaga, Justin;Sweetwood, Kevin;Ye, Allen;Patel, Rina;McGill, Kevin;Link, Thomas;Crane, Jason;Pedoia, Valentina;Majumdar, Sharmila
关键词：
今天，放射科医生在做出诊断决定和标记图像以训练和基准人工智能(AI)算法方面发挥着核心作用。一个关键的问题是，在解释具有挑战性的案例时，专家之间的读者间可靠性(IRR)较低。虽然众所周知，基于团队的决策优于个人决策，但在群体互动中，个人之间的偏见经常悄悄出现，限制了非主导参与者表达真实观点。为了克服共识低和人际偏见的双重问题，我们探索了一种模仿蜂群的解决方案。两个独立的队列，三个委员会认证的放射科医生(队列1)和五个放射科住院医生(队列2)在一个数字群平台上以盲目的方式实时协作，在膝盖MR检查中对半月板损伤进行分级。这些共识投票以临床(关节镜)和放射学(最高级放射科医生)标准为基准，使用科恩的kappa。然后，将协商一致投票的内部收益率与两组多数和最有信心的选票的内部收益率进行比较。对于半月板损伤检测AI算法的预测，也计算了IRR。参加队列的群体投票(k=0.34)的IRR比多数投票(k=0.11)提高了23%。在3居民群体投票(k=0.25)中，内部回报率(IRR)比多数投票(k=0.02)也有类似的提高。5人蜂群的IRR(k=0.37)比多数票(k=0.07)有更高的改善。在放射科医生和居民队列中，群体共识投票的表现都超过了个人和多数人的投票决定。与会者和居民蜂群的表现也超过了最先进的人工智能算法的预测。网上版载有补充材料，可在10.1007/s10278-022-00662-3查阅。
Radiologists today play a central role in making diagnostic decisions and labeling images for training and benchmarking artificial intelligence (AI) algorithms. A key concern is low inter-reader reliability (IRR) seen between experts when interpreting challenging cases. While team-based decisions are known to outperform individual decisions, inter-personal biases often creep up in group interactions which limit nondominant participants from expressing true opinions. To overcome the dual problems of low consensus and interpersonal bias, we explored a solution modeled on bee swarms. Two separate cohorts, three board-certified radiologists, (cohort 1), and five radiology residents (cohort 2) collaborated on a digital swarm platform in real time and in a blinded fashion, grading meniscal lesions on knee MR exams. These consensus votes were benchmarked against clinical (arthroscopy) and radiological (senior-most radiologist) standards of reference using Cohen’s kappa. The IRR of the consensus votes was then compared to the IRR of the majority and most confident votes of the two cohorts. IRR was also calculated for predictions from a meniscal lesion detecting AI algorithm. The attending cohort saw an improvement of 23% in IRR of swarm votes (k = 0.34) over majority vote (k = 0.11). Similar improvement of 23% in IRR (k = 0.25) in 3-resident swarm votes over majority vote (k = 0.02) was observed. The 5-resident swarm had an even higher improvement of 30% in IRR (k = 0.37) over majority vote (k = 0.07). The swarm consensus votes outperformed individual and majority vote decision in both the radiologists and resident cohorts. The attending and resident swarms also outperformed predictions from a state-of-the-art AI algorithm. The online version contains supplementary material available at 10.1007/s10278-022-00662-3.
登录
查看更多内容
影响因子：
4.8
作者：
Dunn, WR;Wolf, BR;Spindler, KP
通讯作者：
Spindler, KP
影响因子：
10.9
作者：
Karimi, Davood;Dou, Haoran;Gholipour, Ali
通讯作者：
Gholipour, Ali
影响因子：
3.5
作者：
Brennan, Allison A.;Enns, James T.
通讯作者：
Enns, James T.
DOI：
10.1007/s10606-009-9094-y
发表时间：
2009-08-01
影响因子：
2.4
作者：
Kane, Bridget;Luz, Saturnino
通讯作者：
Luz, Saturnino
影响因子：
3.9
作者：
Kim SH;Lee HJ;Jang YH;Chun KJ;Park YB
通讯作者：
Park YB