Real-Time Pixel-Level Semantic Tracking in Retinal Microsurgery

博士项目目标：该项目将通过创新开发视网膜显微外科再生治疗的深度学习模型：通过光流进行视网膜跟踪，用于仪器分割的多目标学习，用于注射引导的图像稳定。最终，博士学位的成果将用于指导视网膜下视力恢复再生疗法的实施。摘要：基因疗法和细胞疗法正在成为导致失明的严重视网膜疾病的变革性再生疗法。将这些治疗方法输送到特定而脆弱的视网膜组织层（有些薄至10-20微米），需要显微外科手术的精度，达到或超过人类感知和灵巧的极限。因此，视网膜组织需要通过高分辨率成像可视化，并灵巧地操纵以优化结果。此外，组织和工具应实时识别和跟踪，以实现稳定的注射，尽管生理或外科医生诱导患者的运动。该博士项目将研究在玻璃体视网膜显微手术中获得的图像的像素级跟踪和语义解释的无监督学习方法，以关键帧识别、闭环和不确定性估计的形式结合时空限制。学生将创建一个深度学习框架，在几分钟内稳健地跟踪注射点、工具和解剖标志，以确保以安全的流速长时间注射治疗。图像是通过立体生物显微镜获得的，该显微镜可以将视网膜可视化，从而也可以利用左/右一致性。

Aim of the PhD Project:This project will develop deep learning models regenerative therapy delivery in retinal microsurgery by innovating on:Retinal tracking via optical flowMulti-objective learning for instrument segmentationImage stabilisation for injection guidance.Ultimately, the outputs of the PhD will be used to guide the subretinal delivery of sight-restoring regenerative therapies.Summary:Gene therapy and cellular therapy are emerging as transformative regenerative treatments for severe retinal diseases leading to blindness. The delivery of these treatments into specific and delicate retinal tissue layers, some as thin as 10-20um, requires microsurgical precision that lies at and above the limit of human perception and dexterity. As such, retinal tissues need to be visualised via high-resolution imaging and dexterously manipulated to optimise outcomes. In addition, tissue and tools should be recognised and tracked in real time to achieve stable injections despite physiological or surgeon induced patient motions.This PhD project will research unsupervised learning methods for pixel-level tracking and semantic interpretation of images acquired during vitreoretinal microsurgery, incorporating spatiotemporal constraints in the form of keyframe identification, loop-closure, and uncertainty estimation. The student will create a deep learning framework that robustly tracks injection point, tools, and anatomical landmarks over several minutes, to ensure prolonged injection of therapies with safe flow rates. Images are acquired via stereo biomicroscopy that visualises the retina en face, allowing for left/right consistency to also be leveraged.