Context Aware Augmented Reality for Endonasal Endoscopic Surgery

该项目旨在开发工具，以指导外科医生在手术中去除脑垂体上的癌症。进入脑垂体是困难的，目前的一种方法是鼻内入路，通过鼻子。然而，虽然这种方法是微创的，对患者更好，但对外科医生来说在技术上是具有挑战性的。对于外科医生来说，操作工具很困难，但对于外科医生来说，保持上下文意识、记住关键结构的位置和识别也很困难。一种建议的解决方案是结合术前扫描数据，如磁共振成像（MRI）或计算机断层扫描（CT）扫描的信息，并将它们与视频结合使用。通常情况下，工程师们提出了“增强现实”，即MRI/CT扫描的信息简单地覆盖在内窥镜视频的顶部。但是这种方法并没有得到临床团队的青睐，而且结果往往是令人困惑和难以使用的。在这个项目中，我们组建了一个由外科医生和工程师组成的团队，从头开始重新思考增强现实范式。首先，目标是确定在操作的每个阶段在屏幕上显示的最相关的信息。然后，机器学习将用于分析内窥镜视频，并自动识别外科医生正在进行的手术的哪个阶段。然后制导系统将自动切换模式，并为程序的每个阶段提供最有用的信息。最后，我们将使用机器学习技术自动将术前数据与内窥镜视频对齐。最终的结果应该比目前最先进的方法更准确，更具有临床相关性，并且代表了在颅底手术中图像引导性能的真正的一步变化。

This project aims to develop tools to guide a surgeon during surgery to remove cancers on the pituitary gland. Access to the pituitary gland is difficult, and one current approach is the endonasal approach, through the nose. However, while this approach is minimally invasive which is better for the patient, it is technically challenging for the surgeon. It is difficult for the surgeon to manoeuvre the tools, but also difficult for the surgeon to maintain contextual awareness and remember the location of and identify critical structures. One proposed solution is to combine pre-operative scan data, such as information from Magnetic Resonance Imaging (MRI), or Computed Tomography (CT) scans, and use them in conjunction with the video. Typically, engineers have proposed "Augmented Reality", where the information from MRI/CT scans is simply overlaid on top of the endoscopic video. But this approach has not found favour with clinical teams, and the result is often confusing and difficult to use.In this project we have assembled a team of surgeons and engineers to re-think the Augmented Reality paradigm from the ground up. First, the aim is to identify the most relevant information to display on-screen at each stage of the operation. Then machine learning will be used to analyse the endoscopic video, and automatically identify which stage of the procedure the surgeon is working on. The guidance system will then automatically switch modes, and provide the most useful information for each stage of the procedure. Finally, we will automate the alignment of pre-operative data to the endoscopic video, using machine learning techniques.The end result should be more accurate, and more clinically relevant than the current state of the art methods, and represent a genuine step change in performance for image-guidance during skull-base procedures.