Research on robot-assisted minimally invasive surgical device with 3D image navigation

1)癌症是世界各国的主要死因之一，也是延长预期寿命的重大障碍，估计2020年新发癌症病例约为1930万例，癌症死亡人数近1000万人。机器人辅助微创手术是利用现代医疗器械穿过人体表面的微小伤口进行手术，识别癌症的早期征兆，切除肿瘤组织的一种新型手术技术。机器人辅助微创手术由于其术后并发症风险低、术后恢复时间短和创伤小等临床重要性而成为主流。2）目的和目标然而，通过内窥镜视频流的二维（2D）视觉显示找到目标并执行复杂手术偶尔会导致定向障碍。为此，本研究将致力于解决两大挑战，以助推机器人辅助微创手术的发展。挑战1：低质量的内窥镜图像照明不良通常会导致低质量的内窥镜图像，这将影响计算机对手术环境的确定。挑战2：手术器械定位和分割机器人辅助微创手术器械包括内窥镜摄像机和手术工具。术中分割和器械定位是内窥镜导航的重要组成部分。然而，由于人体成像环境差，特别是软组织变形，图像引导的内窥镜导航在准确性和应用方面仍有很大的改进空间。3）研究方法的新奇本研究将设计一种机器人辅助微创手术装置，该装置由布置在微发光二极管周围的单目摄像机组成，可以安装在手术工具上，为机器人提供3D场景导航。辅助微创手术。已经开发出一种可以重建3D场景的可吞咽胶囊内窥镜，其可以在手术前由患者吞咽，并且重建患者身体的三维视图，以便外科医生可以定位病变。结合SLAM技术，手术器械可以真实的实时定位在患者体内。整个过程的照明总是可以通过使用实时图像数据的微发光二极管的亮度值矩阵来调整，以实现均匀的照明。(1)SLAM和仪器分割的组合该系统分为2个并行线程。一个线程用于估计摄像机姿态和场景的变形，而另一个线程应用于手术工具的分割和位置估计，以更好地拟合用于手术导航的内腔变形场景。(2)环境自适应照明LED阵列LED阵列的亮度调节适用于实现均匀照明。可植入和无线控制的微型LED设备提供了解决照明不良问题的潜在方法。4）与EPSRC的战略和研究领域保持一致这项工作旨在使用计算机视觉技术、人工智能、嵌入式开发技术等来解决机器人手术中的工程挑战，这与EPSRC健康技术战略相匹配，称为机器人手术的微创自主技术。这项工作旨在使用计算机视觉，人工智能和嵌入式开发技术来解决机器人手术中的工程挑战，这与EPSRC健康技术战略中的辅助技术和手术机器人等物理干预的创新技术保持一致。

1) Brief description of the context of the research including potential impactCancer is one of the leading causes of death in countries around the world and a significant barrier to increasing life expectancy, an estimated 19.3 million new cancer cases and almost 10.0 million cancer deaths occurred in 2020. Robotic-assisted minimally invasive surgery is a new surgical technique that uses modern medical instruments to pass through small wounds on the human body surface to perform operation, identifying the early signs of cancer and removing tumour tissues. Robot-assisted minimally invasive surgery has become mainstream because of its clinical importance in terms of low risk of post-operative complications, short post-operative recovery time and small wounds.2) Aims and ObjectivesHowever, finding the target and performing complex surgery through a two-dimensional (2D) visual display of an endoscopic video stream would occasionally lead to disorientation. To this end, this study will be committed to addressing two challenges to boost the robot-assisted minimally invasive surgery.Challenge 1: Low-quality endoscopic imagesThe poor illumination usually results in low-quality endoscopic images, which will affect the determination of the surgical environment by the computer.Challenge 2: Surgical instruments localization and segmentationRobot-assisted minimally invasive surgical instruments include endoscopic cameras and surgical tools. Intraoperative segmentation and localization of instruments are very important components of endoscopic navigation. However, owing to the poor imaging environment in the human body, especially the soft tissue deforming, image guided endoscopic navigation still has much room for improvement in accuracy and application.3) Novelty of Research MethodologyThis study will design a robot-assisted minimally invasive surgical device consisting of a monocular camera arranged around micro-LEDs which can be mounted on a surgical tool to provide 3D scene navigation for robot-assisted minimally invasive surgery. A swallowable capsule endoscopy that can reconstruct 3D scene has been developed, which can be swallowed by the patient prior to surgery and a three-dimensional view of the patient's body is reconstructed so that the surgeon can locate the lesion. In combination with SLAM technology, the surgical instruments are positioned inside the patient in real time. The illumination of the entire procedure can always be adjusted by a matrix of brightness values of the micro-LEDs using real-time image data to achieve uniform illumination.(1) Combination of SLAM and instrument segmentationThis system is split into 2 parallel threads. One thread is used to estimate the camera pose and the deformation of the scene, while the other thread is applied to the segmentation and positional estimation of the surgical tools to better fit the internal cavity deformation scene for surgical navigation.(2) Environment adaptive illumination LED arrayAdjustment of brightness of LED array is applicable to implement uniform illumination. Implantable and wireless control micro-LED devices provide potential approaches to solving the problem of poor illumination.4) Alignment to EPSRC's strategies and research areasThis work aims to use computer vision technology, artificial intelligence, embedded development technology etc. to address engineering challenges in robotic surgery which matches the EPSRC Health Technologies Strategy called minimally invasive autonomous technologies for robotic surgery. This work aims to use computer vision, artificial intelligence, and embedded development techniques to address engineering challenges in robotic surgery, which aligns with the innovative technologies for physical intervention such as assistive technologies and surgical robotics in EPSRC Health Technologies Strategy.5) Any companies or collaboratorsNo companies or collaborators.