Virtual reality for synergistic surgical training and data generation
Virtual reality for synergistic surgical training and data generation
复制标题
用于协同手术训练和数据生成的虚拟现实
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
M. Unberath
中科院分区：
文献类型：
--
作者：
A. Munawar;Zhaoshuo Li;Punit Kunjam;N. Nagururu;Andy S Ding;P. Kazanzides;T. Looi;Francis X Creighton;Russell H. Taylor;M. Unberath
摘要 手术模拟器不仅可以规划和训练复杂的手术，还可以生成用于算法开发的结构化数据，这可以应用于图像引导的计算机辅助干预。尽管人们在开发外科医生培训平台或数据生成引擎方面做出了努力，但据我们所知，这两个功能尚未同时提供。我们展示了我们开发的一种经济高效且协同的框架，名为异步多体框架 Plus (AMBF+)，它可以在用户练习手术技能的同时生成用于下游算法开发的数据。 AMBF+ 在虚拟现实 (VR) 设备上提供立体显示，并为沉浸式手术模拟提供触觉反馈。它还可以生成各种数据，例如对象姿势和分割图。 AMBF+ 设计有灵活的插件设置，可以进行不显眼的扩展，以模拟不同的外科手术。我们展示了 AMBF+ 作为横向颅底手术虚拟钻孔模拟器的一个用例，用户可以使用虚拟手术钻主动修改患者的解剖结构。我们进一步演示了如何使用生成的数据来验证和训练下游计算机视觉算法。
ABSTRACT Surgical simulators not only allow planning and training of complex procedures, but also offer the ability to generate structured data for algorithm development, which may be applied in image-guided computer assisted interventions. While there have been efforts on either developing training platforms for surgeons or data generation engines, these two features, to our knowledge, have not been offered together. We present our developments of a cost-effective and synergistic framework, named Asynchronous Multibody Framework Plus (AMBF+), which generates data for downstream algorithm development simultaneously with users practicing their surgical skills. AMBF+ offers stereoscopic display on a virtual reality (VR) device and haptic feedback for immersive surgical simulation. It can also generate diverse data such as object poses and segmentation maps. AMBF+ is designed with a flexible plugin setup that allows for unobtrusive extension for simulation of different surgical procedures. We show one use case of AMBF+ as a virtual drilling simulator for lateral skull-base surgery, where users can actively modify the patient anatomy using a virtual surgical drill. We further demonstrate how the data generated can be used for validating and training downstream computer vision algorithms.
影响因子：
5.2
作者：
Munawar, Adnan;Wu, Jie Ying;Fischer, Gregory S.
通讯作者：
Fischer, Gregory S.