Effective Visualization for Image-guided Interventions

增强现实（AR），其中计算模型与现实世界的数据合并，以提供增强的周围环境的可视化，正在从建筑，游戏，教育，体育和导航的许多领域中找到用途。AR有很大（尚未实现）前景的一个领域是在医学上引导微创治疗或药物输送到体内器官。在该应用中，不是在开放手术期间手动接近暴露的目标，而是借助于通过患者皮肤中的端口放置的工具接近它们。微创介入治疗（MII）的基本原理是，在许多情况下，入路对患者造成的创伤比治疗本身更多。 医学成像技术的最新改进已经推动了微创介入技术的发展，其具有与患者配准的器官或目标的虚拟表示，增强来自内窥镜相机、超声换能器、在手术之前获取的CT或MRI的图像，或者上述的各种组合。使用虚拟信息来增强术中图像视图是这些程序中的有利因素。该提案旨在开发和展示适当的AR策略，这些策略直观易用，并且能够以直观且用户可接受的方式实现新的微创方法。 AR、虚拟现实（VR）和增强虚拟（AV）（其中虚拟模型与可能不会直接重建真实视觉环境的额外数据融合）都在辅助介入手术中发挥了作用，但尚未成为主流手术的一部分。本研究的目的是证明，在直观的用户界面中适当使用VR/AR/AV技术，可以显着降低用户的认知负荷，并显着提高操作员在微创手术中的表现。这些想法在这里描述了三个代表性的项目，包括腹腔镜手术，心脏手术和神经外科，在正在进行的实验室活动的频谱。

Augmented reality (AR), where computed models are merged with real-world data to provide enhanced visualization of one’s surroundings, is finding use in many area from architecture, gaming, education, sports and navigation. One domain where AR has a great deal of (as yet unrealized) promise, is in medicine to guide minimally-invasive therapy or drug delivery to organs inside the body. In this application, instead of exposed targets being accessed manually during an open procedure, they are approached by means of tools placed through ports in the patient’s skin. The rationale for minimally-invasive interventions (MII) is that in many cases more trauma is caused to the patient by the access route than by the therapy itself. Recent improvements in medical imaging technology have motivated the development of minimally-invasive intervention techniques, with virtual representations of organs or targets, registered to the patient, augmenting images from an endoscopic camera, an ultrasound transducer, CT or MRI acquired prior to the procedure, or various combinations of the above. The use of virtual information to augment the intra-operative image view has been the enabling factor in these procedures. This proposal seeks to develop and demonstrate appropriate AR strategies that are intuitive to use and that will enable new minimally-invasive approaches to be realized in a manner that is intuitive and acceptable to the user. AR, Virtual Reality (VR), and Augmented Virtuality (AV) where a virtual model is fused with additional data that may not directly re-create a realistic visual environment, have all played a part assisting interventional procedures but have not become part of mainstream surgery. The aim of this research is to demonstrate that the use of VR/AR/AV techniques, appropriately employed in an intuitive user interface, significantly lowers the cognitive load of the user, and significantly increases operator performance for minimally-invasive procedures. These ideas are described here for three representative projects, including Laparoscopic surgery, Cardiac surgery and Neurosurgery, among the spectrum of ongoing lab activities.