Research on robot-assisted minimally invasive surgical device with 3D image navigation

1)研究背景的简要描述，包括潜在影响癌症是世界各国死亡的主要原因之一，也是提高预期寿命的重要障碍，估计有1930万新的癌症病例和近1000万癌症死亡发生在2020年。机器人辅助微创手术是利用现代医疗器械穿过人体表面的小伤口进行手术，识别癌症的早期迹象并切除肿瘤组织的一种新的外科技术。机器人辅助微创手术因其术后并发症风险低、术后恢复时间短、伤口小等优点在临床上具有重要意义，已成为微创手术的主流。然而，通过内窥镜视频流的二维（2D）视觉显示来寻找目标并进行复杂的手术有时会导致定向障碍。为此，本研究将致力于解决两个挑战，以促进机器人辅助微创手术的发展。挑战1：内镜图像质量低光照差通常会导致内镜图像质量低，影响计算机对手术环境的判断。挑战2：手术器械的定位和分割机器人辅助的微创手术器械包括内窥镜相机和手术工具。术中器械的分割和定位是内镜导航的重要组成部分。然而，由于人体成像环境恶劣，尤其是软组织的变形，图像引导的内镜导航在精度和应用上仍有很大的提升空间。3)研究方法的新颖性本研究将设计一种机器人辅助微创手术装置，该装置由围绕微型led排列的单目摄像头组成，可安装在手术工具上，为机器人辅助微创手术提供三维场景导航。已经开发出一种可以重建三维场景的可吞咽胶囊内窥镜，可以在手术前将其吞下，重建患者身体的三维视图，以便外科医生定位病变。结合SLAM技术，手术器械可以实时定位在患者体内。整个过程的照明始终可以通过使用实时图像数据的微型led亮度值矩阵来调整，以实现均匀照明。(1) SLAM与仪器分割相结合本系统分为2个并行线程。一个线程用于估计摄像机姿态和场景变形，另一个线程用于手术工具的分割和位置估计，以更好地拟合手术导航的内腔变形场景。(2)环境自适应照明LED阵列LED阵列的亮度调节适用于实现均匀照明。植入式和无线控制微型led器件为解决照明不良问题提供了潜在的途径。4)与EPSRC的战略和研究领域保持一致。这项工作旨在使用计算机视觉技术、人工智能、嵌入式开发技术等来解决机器人手术中的工程挑战，这与EPSRC健康技术战略（称为机器人手术的微创自主技术）相匹配。这项工作旨在利用计算机视觉、人工智能和嵌入式开发技术来解决机器人手术中的工程挑战，这与EPSRC健康技术战略中物理干预的创新技术（如辅助技术和手术机器人）保持一致。

1) Brief description of the context of the research including potential impactCancer is one of the leading causes of death in countries around the world and a significant barrier to increasing life expectancy, an estimated 19.3 million new cancer cases and almost 10.0 million cancer deaths occurred in 2020. Robotic-assisted minimally invasive surgery is a new surgical technique that uses modern medical instruments to pass through small wounds on the human body surface to perform operation, identifying the early signs of cancer and removing tumour tissues. Robot-assisted minimally invasive surgery has become mainstream because of its clinical importance in terms of low risk of post-operative complications, short post-operative recovery time and small wounds.2) Aims and ObjectivesHowever, finding the target and performing complex surgery through a two-dimensional (2D) visual display of an endoscopic video stream would occasionally lead to disorientation. To this end, this study will be committed to addressing two challenges to boost the robot-assisted minimally invasive surgery.Challenge 1: Low-quality endoscopic imagesThe poor illumination usually results in low-quality endoscopic images, which will affect the determination of the surgical environment by the computer.Challenge 2: Surgical instruments localization and segmentationRobot-assisted minimally invasive surgical instruments include endoscopic cameras and surgical tools. Intraoperative segmentation and localization of instruments are very important components of endoscopic navigation. However, owing to the poor imaging environment in the human body, especially the soft tissue deforming, image guided endoscopic navigation still has much room for improvement in accuracy and application.3) Novelty of Research MethodologyThis study will design a robot-assisted minimally invasive surgical device consisting of a monocular camera arranged around micro-LEDs which can be mounted on a surgical tool to provide 3D scene navigation for robot-assisted minimally invasive surgery. A swallowable capsule endoscopy that can reconstruct 3D scene has been developed, which can be swallowed by the patient prior to surgery and a three-dimensional view of the patient's body is reconstructed so that the surgeon can locate the lesion. In combination with SLAM technology, the surgical instruments are positioned inside the patient in real time. The illumination of the entire procedure can always be adjusted by a matrix of brightness values of the micro-LEDs using real-time image data to achieve uniform illumination.(1) Combination of SLAM and instrument segmentationThis system is split into 2 parallel threads. One thread is used to estimate the camera pose and the deformation of the scene, while the other thread is applied to the segmentation and positional estimation of the surgical tools to better fit the internal cavity deformation scene for surgical navigation.(2) Environment adaptive illumination LED arrayAdjustment of brightness of LED array is applicable to implement uniform illumination. Implantable and wireless control micro-LED devices provide potential approaches to solving the problem of poor illumination.4) Alignment to EPSRC's strategies and research areasThis work aims to use computer vision technology, artificial intelligence, embedded development technology etc. to address engineering challenges in robotic surgery which matches the EPSRC Health Technologies Strategy called minimally invasive autonomous technologies for robotic surgery. This work aims to use computer vision, artificial intelligence, and embedded development techniques to address engineering challenges in robotic surgery, which aligns with the innovative technologies for physical intervention such as assistive technologies and surgical robotics in EPSRC Health Technologies Strategy.5) Any companies or collaboratorsNo companies or collaborators.