Development of an augmented reality image-guidance system for cardiac interventions

每年有7万加拿大人心脏病发作，导致1.6万人死亡。心脏病发作会留下一块无功能的疤痕(或死亡)组织，导致心脏功能不佳，最终导致心力衰竭。不幸的是，丢失的细胞不能再生。通过移植新的功能心脏细胞来重新填充疤痕组织的能力代表了这些患者的一种革命性的新疗法。新的细胞可以通过心内直视手术进入心脏，外科医生将细胞直接注射到心肌中。然而，在外科手术过程中，外科医生无法在视觉上看到疤痕组织的整个位置和范围，从而能够将细胞输送到正确的位置，从而失去治疗的有效性。我们的研究计划提供了一种新的解决方案，使用图像引导技术来克服细胞输送中的重要障碍。我们的目标是建立一个系统，将心脏磁共振成像(MRI)产生的疤痕的高分辨率三维(3D)路线图与新兴的增强现实(AR)技术相结合。我们将使用头戴式AR设备，允许用户与放置在生活3D空间中的虚拟对象进行交互。我们将首先建立和校准一个AR框架，在该框架中，用户将能够使用手势和语音命令来操作全息图。我们将获得高分辨率的3D MRI疤痕图，并将它们作为3D全息图投影到用户的视野中。该系统将能够对齐3D全息图，使用智能标记将它们正确地放置在目标区域上，并进一步自动调整它们的形状和大小，以补偿跳动的心脏和患者的呼吸动作。该系统将得到增强，用于多AR设备通信，并允许使用外部图形处理单元快速处理复杂的心脏运动。该系统还将预测预先计划的注射地点，以使外科医生的工作流程高效和优化。在心脏手术中基于增强现实技术的图像引导是一个未被探索的领域。该项目开发的集成成像技术将是此类技术中的第一项，并将为再生医学研究人员提供一个独特而强大的框架。重要的是，这一系统对人类是直接可翻译的，该计划将使我们能够将这些新技术部署到涉及新细胞系和用于治疗心力衰竭的先进生物材料的高影响关键疗效研究中。此外，加拿大的设备和生物技术合作伙伴可以利用这些图像引导技术实现商业化，并加快加拿大医疗系统的供应。我们相信，拟议的AR框架也将对其他心脏干预措施产生影响，包括开胸手术搭桥、心脏电生理消融和结构性心脏缺陷。

Each year 70,000 Canadians suffer a heart attack resulting in 16,000 deaths. A heart attack leaves behind a region of non-functional scar (or dead) tissue, resulting in a poorly functioning heart and ultimately heart failure. Unfortunately, the cells that are lost cannot be regenerated. The ability to repopulate the scar tissue by transplanting new functional heart cells represents a revolutionary new therapy for these patients. The new cells can be delivered into the heart through an open-heart surgery where a surgeon will inject the cells directly into the heart muscle. However, during a surgical procedure, the surgeon is unable to visually "see" the entire location and extent of the scar tissue to be able to deliver the cells in the right place, in turn losing the effectiveness of the therapy. Our research program offers a novel solution using image guidance technologies to overcome important hurdles in cell delivery. We aim to build a system that integrates high resolution three-dimensional (3D) roadmaps of the scar generated from cardiac Magnetic Resonance Imaging (MRI) with the emerging technology of augmented reality (AR). We will use head mounted AR devices that allow the user to interact with virtual objects placed in the living 3D space. We will first establish and calibrate an AR framework where users will be able to manipulate the holograms using hand gestures and voice commands. We will obtain high resolution 3D MRI scar maps and project them as 3D holograms into the user's visual field. The system will be able to align the 3D holograms to correctly place them over the target area using smart markers and further automatically adjust their shape and size to compensate for the movement of the beating heart and patient's breathing action. The system will be enhanced for multi-AR device communication and allow for rapid processing of complex heart motion with an external graphics processing unit. The system will also project preplanned injection sites to make the surgeon's workflow efficient and optimal. Augmented reality technology-based image-guidance in cardiac surgery is an unexplored area. The integrated imaging technologies developed in this program will be a first of its kind and will offer a unique and powerful framework for regenerative medicine investigators. Importantly, this system is directly translatable to humans and the program will position us towards deploying these novel technologies into high-impact pivotal efficacy studies involving new cell lines and advanced biomaterials for the treatment of heart failure. Furthermore, these image-guidance technologies can be leveraged by Canada-based devices and biotechnology partners for commercialization and accelerated availability in the Canadian healthcare system. We believe that the proposed AR framework will have implications in other cardiac interventions as well including open chest surgery for bypass, cardiac electrophysiology ablation and structural heart defects.