Effective Visualization for Image-guided Interventions

增强现实技术（AR）将计算模型与真实世界的数据相结合，以增强对周围环境的可视化，在建筑、游戏、教育、体育和导航等许多领域都有应用。增强现实在医学上有很大的前景（尚未实现），它可以引导微创治疗或将药物输送到体内器官。在此应用中，在开放手术过程中，不是手动访问暴露的目标，而是通过通过患者皮肤中的端口放置的工具接近目标。微创干预（MII）的基本原理是，在许多情况下，接触途径比治疗本身对患者造成的创伤更大。医学成像技术的最新进步推动了微创介入技术的发展，包括器官或目标的虚拟表示，注册到患者身上，增强内窥镜相机、超声换能器、手术前获得的CT或MRI图像，或上述技术的各种组合。使用虚拟信息来增强术中图像视图是这些手术的有利因素。本提案旨在开发和演示适当的AR策略，这些策略直观易用，并将使新的微创方法能够以直观和用户可接受的方式实现。AR，虚拟现实（VR）和增强虚拟现实（AV），其中虚拟模型与可能无法直接重建现实视觉环境的附加数据融合在一起，都在辅助介入手术中发挥了作用，但尚未成为主流手术的一部分。本研究的目的是证明，在直观的用户界面中适当使用VR/AR/AV技术，可以显著降低用户的认知负荷，并显著提高微创手术的操作性能。这些想法在这里描述了三个代表性的项目，包括腹腔镜手术，心脏手术和神经外科，在正在进行的实验室活动的频谱。

Augmented reality (AR), where computed models are merged with real-world data to provide enhanced visualization of one’s surroundings, is finding use in many area from architecture, gaming, education, sports and navigation. One domain where AR has a great deal of (as yet unrealized) promise, is in medicine to guide minimally-invasive therapy or drug delivery to organs inside the body. In this application, instead of exposed targets being accessed manually during an open procedure, they are approached by means of tools placed through ports in the patient’s skin. The rationale for minimally-invasive interventions (MII) is that in many cases more trauma is caused to the patient by the access route than by the therapy itself. Recent improvements in medical imaging technology have motivated the development of minimally-invasive intervention techniques, with virtual representations of organs or targets, registered to the patient, augmenting images from an endoscopic camera, an ultrasound transducer, CT or MRI acquired prior to the procedure, or various combinations of the above. The use of virtual information to augment the intra-operative image view has been the enabling factor in these procedures. This proposal seeks to develop and demonstrate appropriate AR strategies that are intuitive to use and that will enable new minimally-invasive approaches to be realized in a manner that is intuitive and acceptable to the user. AR, Virtual Reality (VR), and Augmented Virtuality (AV) where a virtual model is fused with additional data that may not directly re-create a realistic visual environment, have all played a part assisting interventional procedures but have not become part of mainstream surgery. The aim of this research is to demonstrate that the use of VR/AR/AV techniques, appropriately employed in an intuitive user interface, significantly lowers the cognitive load of the user, and significantly increases operator performance for minimally-invasive procedures. These ideas are described here for three representative projects, including Laparoscopic surgery, Cardiac surgery and Neurosurgery, among the spectrum of ongoing lab activities.