Research on robot-assisted minimally invasive surgical device with 3D image navigation

1)简要说明研究背景，包括潜在影响癌症是世界各国的主要死亡原因之一，也是提高预期寿命的重大障碍，2020年估计有1930万新癌症病例和近1000万癌症死亡病例。机器人辅助微创手术是一种新的外科技术，它利用现代医疗器械穿过人体表面的小伤口进行手术，识别癌症的早期迹象，并切除肿瘤组织。机器人辅助微创手术因其术后并发症风险低、术后恢复时间短、创伤小等临床重要性而成为主流。2)目的和目的然而，通过内窥镜视频流的二维(2D)视觉显示来寻找目标并执行复杂的手术有时会导致定向障碍。为此，本研究将致力于解决两个挑战，以促进机器人辅助的微创手术。挑战1：低质量的内窥镜图像照明通常会导致低质量的内窥镜图像，这将影响计算机对手术环境的确定。挑战2：手术器械的定位和分割机器人辅助的微创手术器械包括内窥镜摄像头和手术工具。术中器械的分割和定位是内窥镜导航的重要组成部分。然而，由于人体成像环境较差，尤其是软组织的变形，图像引导的内窥镜导航在精度和应用上仍有很大的提高空间。3)研究方法的新颖性本研究将设计一种机器人辅助微创手术装置，该装置由围绕微LED的单目摄像头组成，可以安装在手术工具上，为机器人辅助微创手术提供3D场景导航。一种可以重建3D场景的可吞咽胶囊内窥镜已经开发出来，患者可以在手术前吞咽，并重建患者身体的三维视图，以便外科医生定位病变。与SLAM技术相结合，手术器械实时定位在患者体内。整个程序的照度始终可以通过使用实时图像数据的微型LED的亮度值矩阵来调节，以实现均匀的照明。(1)SLAM和仪器分割的组合本系统分为两个平行的线程。一个线程用于估计摄像机的姿态和场景的变形，另一个线程用于手术工具的分割和位置估计，以更好地适应手术导航中的内腔变形场景。(2)环境自适应照明LED阵列适用于调整LED阵列的亮度，以实现均匀照明。可植入和无线控制的微型LED器件为解决照明不足的问题提供了潜在的方法。4)与EPSRC的战略和研究领域保持一致这项工作旨在利用计算机视觉技术、人工智能、嵌入式开发技术等来应对机器人手术中的工程挑战，该挑战与EPSRC健康技术战略相匹配，称为机器人手术的微创自主技术。这项工作旨在使用计算机视觉、人工智能和嵌入式开发技术来解决机器人手术中的工程挑战，这与EPSRC健康技术战略中的辅助技术和手术机器人等物理干预的创新技术保持一致。5)任何公司或合作者没有公司或合作者。

1) Brief description of the context of the research including potential impactCancer is one of the leading causes of death in countries around the world and a significant barrier to increasing life expectancy, an estimated 19.3 million new cancer cases and almost 10.0 million cancer deaths occurred in 2020. Robotic-assisted minimally invasive surgery is a new surgical technique that uses modern medical instruments to pass through small wounds on the human body surface to perform operation, identifying the early signs of cancer and removing tumour tissues. Robot-assisted minimally invasive surgery has become mainstream because of its clinical importance in terms of low risk of post-operative complications, short post-operative recovery time and small wounds.2) Aims and ObjectivesHowever, finding the target and performing complex surgery through a two-dimensional (2D) visual display of an endoscopic video stream would occasionally lead to disorientation. To this end, this study will be committed to addressing two challenges to boost the robot-assisted minimally invasive surgery.Challenge 1: Low-quality endoscopic imagesThe poor illumination usually results in low-quality endoscopic images, which will affect the determination of the surgical environment by the computer.Challenge 2: Surgical instruments localization and segmentationRobot-assisted minimally invasive surgical instruments include endoscopic cameras and surgical tools. Intraoperative segmentation and localization of instruments are very important components of endoscopic navigation. However, owing to the poor imaging environment in the human body, especially the soft tissue deforming, image guided endoscopic navigation still has much room for improvement in accuracy and application.3) Novelty of Research MethodologyThis study will design a robot-assisted minimally invasive surgical device consisting of a monocular camera arranged around micro-LEDs which can be mounted on a surgical tool to provide 3D scene navigation for robot-assisted minimally invasive surgery. A swallowable capsule endoscopy that can reconstruct 3D scene has been developed, which can be swallowed by the patient prior to surgery and a three-dimensional view of the patient's body is reconstructed so that the surgeon can locate the lesion. In combination with SLAM technology, the surgical instruments are positioned inside the patient in real time. The illumination of the entire procedure can always be adjusted by a matrix of brightness values of the micro-LEDs using real-time image data to achieve uniform illumination.(1) Combination of SLAM and instrument segmentationThis system is split into 2 parallel threads. One thread is used to estimate the camera pose and the deformation of the scene, while the other thread is applied to the segmentation and positional estimation of the surgical tools to better fit the internal cavity deformation scene for surgical navigation.(2) Environment adaptive illumination LED arrayAdjustment of brightness of LED array is applicable to implement uniform illumination. Implantable and wireless control micro-LED devices provide potential approaches to solving the problem of poor illumination.4) Alignment to EPSRC's strategies and research areasThis work aims to use computer vision technology, artificial intelligence, embedded development technology etc. to address engineering challenges in robotic surgery which matches the EPSRC Health Technologies Strategy called minimally invasive autonomous technologies for robotic surgery. This work aims to use computer vision, artificial intelligence, and embedded development techniques to address engineering challenges in robotic surgery, which aligns with the innovative technologies for physical intervention such as assistive technologies and surgical robotics in EPSRC Health Technologies Strategy.5) Any companies or collaboratorsNo companies or collaborators.