Pose Estimation and Non-Rigid Registration for Augmented Reality During Neurosurgery.
Pose Estimation and Non-Rigid Registration for Augmented Reality During Neurosurgery.
复制标题
DOI：
10.1109/tbme.2021.3113841
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
中科院分区：
文献类型：
--
作者：
开颅手术是切除颅骨的一部分，使外科医生能够进入大脑并治疗肿瘤。当进入大脑时，会发生组织变形，并可能对外科手术结果产生负面影响。在这项工作中，我们提出了一种新的增强现实神经外科手术系统，将手术前的3D网格从MRI中提取到手术过程中获得的大脑表面的视图上。我们的方法使用皮质血管作为主要特征来驱动刚性然后非刚性3D/2D配准。我们首先使用特征提取器网络来产生概率图，该概率图被馈送到姿态估计器网络以推断6-DoF刚性姿态。然后，考虑到大脑变形，我们添加了一个非刚性的细化步骤，制定为一个形状从模板的问题，使用基于物理的约束，有助于传播变形到皮层下的水平和更新肿瘤的位置。我们在6个临床数据集上回顾性地测试了我们的方法，获得了较低的姿势误差，并使用合成数据集表明，在皮层和皮层下水平可以实现相当大的脑移位补偿和较低的TRE。结果表明，我们的解决方案达到了低于实际临床误差的精度，证明了我们的系统的实际使用的可行性。这项工作表明，我们可以提供连贯的增强现实可视化的3D皮质血管观察通过开颅手术使用一个单一的摄像头视图和皮质血管提供强大的功能，执行刚性和非刚性注册。
A craniotomy is the removal of a part of the skull to allow surgeons to have access to the brain and treat tumors. When accessing the brain, a tissue deformation occurs and can negatively influence the surgical procedure outcome. In this work, we present a novel Augmented Reality neurosurgical system to superimpose pre-operative 3D meshes derived from MRI onto a view of the brain surface acquired during surgery. Our method uses cortical vessels as main features to drive a rigid then non-rigid 3D/2D registration. We first use a feature extractor network to produce probability maps that are fed to a pose estimator network to infer the 6-DoF rigid pose. Then, to account for brain deformation, we add a non-rigid refinement step formulated as a Shape-from-Template problem using physics-based constraints that helps propagate the deformation to sub-cortical level and update tumor location. We tested our method retrospectively on 6 clinical datasets and obtained low pose error, and showed using synthetic dataset that considerable brain shift compensation and low TRE can be achieved at cortical and sub-cortical levels. The results show that our solution achieved accuracy below the actual clinical errors demonstrating the feasibility of practical use of our system. This work shows that we can provide coherent Augmented Reality visualization of 3D cortical vessels observed through the craniotomy using a single camera view and that cortical vessels provide strong features for performing both rigid and non-rigid registration.