Effective Visualization for Image-guided Interventions

增强现实(AR)将计算模型与真实世界的数据结合在一起，提供对周围环境的增强可视化，在建筑、游戏、教育、体育和导航等许多领域都有应用。AR有一个很有希望(尚未实现)的领域，那就是在医学领域，引导微创治疗或药物输送到体内器官。在这一应用中，不是在开放手术期间手动接触暴露的靶子，而是通过放置在患者皮肤上的端口的工具接近它们。微创介入(MII)的基本原理是，在许多情况下，通路给患者造成的创伤比治疗本身造成的更多。**最近医学成像技术的进步推动了微创干预技术的发展，通过注册给患者的器官或靶点的虚拟表示，增强来自内窥镜相机、超声换能器、手术前获得的CT或MRI的图像，或上述各种组合。在这些手术中，使用虚拟信息来增强术中图像的视野一直是使能因素。这项建议旨在开发和演示适当的AR策略，这些策略直观地使用，并将使新的微创方法能够以直观和用户可以接受的方式实现。*AR、虚拟现实(VR)和增强虚拟(AV)将虚拟模型与可能无法直接重建真实视觉环境的额外数据融合在一起，这些都起到了辅助介入手术的作用，但尚未成为主流手术的一部分。这项研究的目的是证明，在直观的用户界面中适当地使用VR/AR/AV技术，可以显著降低用户的认知负荷，并显著提高操作员在微创手术中的表现。这些想法在这里描述了三个具有代表性的项目，包括腹腔镜手术，心脏外科和神经外科，在正在进行的实验室活动范围中。

Augmented reality (AR), where computed models are merged with real-world data to provide enhanced visualization of one's surroundings, is finding use in many area from architecture, gaming, education, sports and navigation. One domain where AR has a great deal of (as yet unrealized) promise, is in medicine to guide minimally-invasive therapy or drug delivery to organs inside the body. In this application, instead of exposed targets being accessed manually during an open procedure, they are approached by means of tools placed through ports in the patient's skin. The rationale for minimally-invasive interventions (MII) is that in many cases more trauma is caused to the patient by the access route than by the therapy itself. **Recent improvements in medical imaging technology have motivated the development of minimally-invasive intervention techniques, with virtual representations of organs or targets, registered to the patient, augmenting images from an endoscopic camera, an ultrasound transducer, CT or MRI acquired prior to the procedure, or various combinations of the above. The use of virtual information to augment the intra-operative image view has been the enabling factor in these procedures. This proposal seeks to develop and demonstrate appropriate AR strategies that are intuitive to use and that will enable new minimally-invasive approaches to be realized in a manner that is intuitive and acceptable to the user. **AR, Virtual Reality (VR), and Augmented Virtuality (AV) where a virtual model is fused with additional data that may not directly re-create a realistic visual environment, have all played a part assisting interventional procedures but have not become part of mainstream surgery. The aim of this research is to demonstrate that the use of VR/AR/AV techniques, appropriately employed in an intuitive user interface, significantly lowers the cognitive load of the user, and significantly increases operator performance for minimally-invasive procedures. These ideas are described here for three representative projects, including Laparoscopic surgery, Cardiac surgery and Neurosurgery, among the spectrum of ongoing lab activities.