Query-by-example surgical activity detection
Query-by-example surgical activity detection
复制标题
DOI：
10.1007/s11548-016-1386-3
复制
发表时间：
2016-06-01
影响因子：
3
通讯作者：
Hager, Gregory D.
中科院分区：
文献类型：
--
作者：
Gao, Yixin;Vedula, S. Swaroop;Hager, Gregory D.
外科数据的容易获取为自动化技能评估和教学提供了许多机会。目前搜索工具运动数据以获取手术活动部分的技术受到人工预处理需求的限制，这在规模上可能是令人望而却步的。我们开发了一种基于内容的信息检索方法，即按例查询（query-by-example， QBE），用于自动检测长时间外科数据记录中与查询匹配的活动片段。感兴趣的样例片段（查询）和手术数据记录（靶试验）是运动学的时间序列。我们的方法包括一个使用堆叠去噪自动编码器（SDAE）的无监督特征学习模块，两个分别基于非对称子序列动态时间规整（AS-DTW）和模板匹配的评分模块，以及一个检测模块。使用SDAE特征计算针对试验的查询的距离矩阵，然后使用AS-DTW结合模板评分，生成候选子序列（子字符串）的排序列表。为了根据真实情况评估排名列表的质量，应用了阈值法、传统的DTW距离和二部匹配。我们在三个实验设置中计算了召回率、准确率、f1分数和基于Jaccard指数的分数。我们在培训实验室捕获的两个工具运动数据集（JIGSAWS和MISTIC-SL）上，使用缝线投掷动作作为查询来评估我们的QBE方法。我们观察到，在JIGSAWS上，同一外科医生同一试验（SSST）、同一外科医生不同试验（SSDT）和不同外科医生（DS）实验设置的召回率分别为93%、90%和87%，精度分别为93%、91%和88%，在misstic - sl上，SSST、SSDT和DS实验设置的召回率分别为87%、81%和75%，精度分别为72,61%和53%。我们开发了一种新颖的，基于内容的信息检索方法来自动检测长手术记录中的多个活动实例。我们的方法在不同的复杂性数据集和实验条件下证明了足够的召回率。
Easy acquisition of surgical data opens many opportunities to automate skill evaluation and teaching. Current technology to search tool motion data for surgical activity segments of interest is limited by the need for manual pre-processing, which can be prohibitive at scale. We developed a content-based information retrieval method, query-by-example (QBE), to automatically detect activity segments within surgical data recordings of long duration that match a query.The example segment of interest (query) and the surgical data recording (target trial) are time series of kinematics. Our approach includes an unsupervised feature learning module using a stacked denoising autoencoder (SDAE), two scoring modules based on asymmetric subsequence dynamic time warping (AS-DTW) and template matching, respectively, and a detection module. A distance matrix of the query against the trial is computed using the SDAE features, followed by AS-DTW combined with template scoring, to generate a ranked list of candidate subsequences (substrings). To evaluate the quality of the ranked list against the ground-truth, thresholding conventional DTW distances and bipartite matching are applied. We computed the recall, precision, F1-score, and a Jaccard index-based score on three experimental setups. We evaluated our QBE method using a suture throw maneuver as the query, on two tool motion datasets (JIGSAWS and MISTIC-SL) captured in a training laboratory.We observed a recall of 93, 90 and 87 % and a precision of 93, 91, and 88 % with same surgeon same trial (SSST), same surgeon different trial (SSDT) and different surgeon (DS) experiment setups on JIGSAWS, and a recall of 87, 81 and 75 % and a precision of 72, 61, and 53 % with SSST, SSDT and DS experiment setups on MISTIC-SL, respectively.We developed a novel, content-based information retrieval method to automatically detect multiple instances of an activity within long surgical recordings. Our method demonstrated adequate recall across different complexity datasets and experimental conditions.