Context Aware Augmented Reality for Endonasal Endoscopic Surgery

该项目旨在开发工具，以指导外科医生在手术中切除脑下垂体上的癌症。进入脑垂体是困难的，目前的一种方法是通过鼻子的鼻内途径。然而，虽然这种方法是微创的，对患者更好，但对外科医生来说在技术上具有挑战性。外科医生很难操纵工具，而且外科医生也很难保持上下文意识并记住关键结构的位置并识别关键结构。一种所提出的解决方案是联合收割机组合术前扫描数据，例如来自磁共振成像（MRI）或计算机断层摄影（CT）扫描的信息，并将它们与视频结合使用。通常，工程师们提出了“增强现实”，其中来自MRI/CT扫描的信息简单地覆盖在内窥镜视频之上。但这种方法并没有得到临床团队的青睐，结果往往是混乱和难以使用。在这个项目中，我们组建了一个由外科医生和工程师组成的团队，从头开始重新思考增强现实模式。首先，目的是确定在操作的每个阶段在屏幕上显示的最相关的信息。然后，机器学习将用于分析内窥镜视频，并自动识别外科医生正在进行的手术阶段。然后，引导系统将自动切换模式，并为手术的每个阶段提供最有用的信息。最后，我们将使用机器学习技术自动将术前数据与内窥镜视频对齐。最终结果应该比当前最先进的方法更准确，更具有临床相关性，并代表了颅底手术期间图像引导性能的真正飞跃。

This project aims to develop tools to guide a surgeon during surgery to remove cancers on the pituitary gland. Access to the pituitary gland is difficult, and one current approach is the endonasal approach, through the nose. However, while this approach is minimally invasive which is better for the patient, it is technically challenging for the surgeon. It is difficult for the surgeon to manoeuvre the tools, but also difficult for the surgeon to maintain contextual awareness and remember the location of and identify critical structures. One proposed solution is to combine pre-operative scan data, such as information from Magnetic Resonance Imaging (MRI), or Computed Tomography (CT) scans, and use them in conjunction with the video. Typically, engineers have proposed "Augmented Reality", where the information from MRI/CT scans is simply overlaid on top of the endoscopic video. But this approach has not found favour with clinical teams, and the result is often confusing and difficult to use.In this project we have assembled a team of surgeons and engineers to re-think the Augmented Reality paradigm from the ground up. First, the aim is to identify the most relevant information to display on-screen at each stage of the operation. Then machine learning will be used to analyse the endoscopic video, and automatically identify which stage of the procedure the surgeon is working on. The guidance system will then automatically switch modes, and provide the most useful information for each stage of the procedure. Finally, we will automate the alignment of pre-operative data to the endoscopic video, using machine learning techniques.The end result should be more accurate, and more clinically relevant than the current state of the art methods, and represent a genuine step change in performance for image-guidance during skull-base procedures.