Measuring to Improve: Peer and Crowd-sourced Assessments of Technical Skill with Robot-assisted Radical Prostatectomy
Measuring to Improve: Peer and Crowd-sourced Assessments of Technical Skill with Robot-assisted Radical Prostatectomy
复制标题
DOI：
10.1016/j.eururo.2015.11.028
复制
发表时间：
2016-04-01
期刊：
影响因子：
23.4
通讯作者：
Peabody, James O.
中科院分区：
文献类型：
--
作者：
Ghani, Khurshid R.;Miller, David C.;Peabody, James O.
由于手术技能可能是患者结局的关键决定因素，因此对技能评估的兴趣越来越大。在密歇根泌尿外科改进协作（MUSIC）中，我们评估了同行和众包（即外行）对机器人辅助根治性膀胱切除术（RARP）的视频审查是否可以区分执业外科医生的技术技能。来自12名MUSIC外科医生的76个视频片段由RARP的四个部分之一组成，并由MUSIC同行外科医生和预先合格的众包评审员进行盲态评审。对视频进行了全局技能（机器人技能的全局评价评估）和程序特定技能（机器人吻合和能力评价）的评级。我们拟合线性混合效应模型来估计每个视频的平均同行和人群评分。汇总单个视频评级以计算外科医生技能评分。同行（n = 25）在15 d内完成了351个视频评级，而众包评审员（n = 680）在38 h内完成了2990个视频评级。外科医生总体技能评分范围为15.8 - 21.7（同行）和19.2 - 20.9（人群）。同伴和人群评级显示出强相关性的全球（r = 0.78）和吻合（r = 0.74）的技能。两组一致同意评分较低的外科医生的排名顺序，这表明众包方法在评估手术性能中的潜在作用。缺乏患者结局是一个限制因素，也是未来研究的基础。患者总结：我们证明了评估机器人外科医生技术技能的大规模可行性，并发现在线众包评审员同意专家对技术技能得分最低的外科医生的排名顺序。出版社：Elsevier B. V.
Because surgical skill may be a key determinant of patient outcomes, there is growing interest in skill assessment. In the Michigan Urological Surgery Improvement Collaborative (MUSIC), we assessed whether peer and crowd-sourced (ie, layperson) video review of robot-assisted radical prostatectomy (RARP) could distinguish technical skill among practicing surgeons. A total of 76 video clips from 12 MUSIC surgeons consisted of one of four parts of RARP and underwent blinded review by MUSIC peer surgeons and prequalified crowd-sourced reviewers. Videos were rated for global skill (Global Evaluation Assessment of Robotic Skills) and procedure-specific skill (Robotic Anastomosis and Competency Evaluation). We fit linear mixed-effects models to estimate mean peer and crowd ratings for each video. Individual video ratings were aggregated to calculate surgeon skill scores. Peers (n = 25) completed 351 video ratings over 15 d, whereas crowd-sourced reviewers (n = 680) completed 2990 video ratings in 38 h. Surgeon global skill scores ranged from 15.8 to 21.7 (peer) and from 19.2 to 20.9 (crowd). Peer and crowd ratings demonstrated strong correlation for both global (r = 0.78) and anastomosis (r = 0.74) skills. The two groups consistently agreed on the rank order of lower scoring surgeons, suggesting a potential role for crowd-sourced methodology in the assessment of surgical performance. Lack of patient outcomes is a limitation and forms the basis of future study. Patient summary: We demonstrated the large-scale feasibility of assessing the technical skill of robotic surgeons and found that online crowd-sourced reviewers agreed with experts on the rank order of surgeons with the lowest technical skill scores. Published by Elsevier B.V.