Development of an augmented reality image-guidance system for cardiac interventions

每年有70,000加拿大人心脏病发作，导致16,000人死亡。心脏病发作后会留下一块无功能的疤痕（或死亡）组织，导致心脏功能不佳，最终导致心力衰竭。不幸的是，失去的细胞不能再生。通过移植新的功能性心脏细胞来重新填充疤痕组织的能力代表了一种革命性的新疗法。这些新细胞可以通过开胸手术进入心脏，外科医生会将细胞直接注射到心肌中。然而，在外科手术过程中，外科医生无法通过视觉“看到”疤痕组织的整个位置和范围，从而无法将细胞输送到正确的位置，从而失去了治疗的有效性。我们的研究计划提供了一种新的解决方案，使用图像引导技术来克服细胞递送中的重要障碍。我们的目标是建立一个系统，将心脏磁共振成像（MRI）产生的疤痕的高分辨率三维（3D）路线图与新兴的增强现实（AR）技术相结合。我们将使用头戴式AR设备，允许用户与放置在生活3D空间中的虚拟物体进行交互。我们将首先建立和校准一个增强现实框架，用户将能够使用手势和语音命令操纵全息图。我们将获得高分辨率的3D MRI疤痕图，并将其作为3D全息图投射到用户的视野中。该系统将能够对齐3D全息图，使用智能标记将它们正确地放置在目标区域上，并进一步自动调整它们的形状和大小，以补偿心脏跳动和患者呼吸动作的运动。该系统将加强多增强现实设备通信，并允许使用外部图形处理单元快速处理复杂的心脏运动。该系统还将预测预先计划的注射位置，使外科医生的工作流程高效和优化。基于增强现实技术的心脏外科图像引导是一个未开发的领域。在这个项目中开发的集成成像技术将是同类技术中的第一个，并将为再生医学研究人员提供一个独特而强大的框架。重要的是，该系统可直接应用于人类，该项目将使我们能够将这些新技术应用于涉及新细胞系和先进生物材料的心力衰竭治疗的高影响力关键疗效研究。此外，这些图像引导技术可以被加拿大的设备和生物技术合作伙伴用于商业化和加速加拿大医疗保健系统的可用性。我们相信，提出的AR框架将对其他心脏干预措施产生影响，包括开胸搭桥手术、心脏电生理消融和结构性心脏缺陷。

Each year 70,000 Canadians suffer a heart attack resulting in 16,000 deaths. A heart attack leaves behind a region of non-functional scar (or dead) tissue, resulting in a poorly functioning heart and ultimately heart failure. Unfortunately, the cells that are lost cannot be regenerated. The ability to repopulate the scar tissue by transplanting new functional heart cells represents a revolutionary new therapy for these patients. The new cells can be delivered into the heart through an open-heart surgery where a surgeon will inject the cells directly into the heart muscle. However, during a surgical procedure, the surgeon is unable to visually "see" the entire location and extent of the scar tissue to be able to deliver the cells in the right place, in turn losing the effectiveness of the therapy. Our research program offers a novel solution using image guidance technologies to overcome important hurdles in cell delivery. We aim to build a system that integrates high resolution three-dimensional (3D) roadmaps of the scar generated from cardiac Magnetic Resonance Imaging (MRI) with the emerging technology of augmented reality (AR). We will use head mounted AR devices that allow the user to interact with virtual objects placed in the living 3D space. We will first establish and calibrate an AR framework where users will be able to manipulate the holograms using hand gestures and voice commands. We will obtain high resolution 3D MRI scar maps and project them as 3D holograms into the user's visual field. The system will be able to align the 3D holograms to correctly place them over the target area using smart markers and further automatically adjust their shape and size to compensate for the movement of the beating heart and patient's breathing action. The system will be enhanced for multi-AR device communication and allow for rapid processing of complex heart motion with an external graphics processing unit. The system will also project preplanned injection sites to make the surgeon's workflow efficient and optimal. Augmented reality technology-based image-guidance in cardiac surgery is an unexplored area. The integrated imaging technologies developed in this program will be a first of its kind and will offer a unique and powerful framework for regenerative medicine investigators. Importantly, this system is directly translatable to humans and the program will position us towards deploying these novel technologies into high-impact pivotal efficacy studies involving new cell lines and advanced biomaterials for the treatment of heart failure. Furthermore, these image-guidance technologies can be leveraged by Canada-based devices and biotechnology partners for commercialization and accelerated availability in the Canadian healthcare system. We believe that the proposed AR framework will have implications in other cardiac interventions as well including open chest surgery for bypass, cardiac electrophysiology ablation and structural heart defects.