EasyVis: Flexible, immersive three-dimensional laparoscopic surgical visualization through multi-camera arrays

摘要 腹腔镜或“微创”手术在当今的外科手术中经常进行。它提供 为患者带来了众多临床益处，并已成为许多患者的黄金标准外科手术。 腹腔和胸腔手术。现有的腹腔镜提供手术区域的可视化 通过一个插入病人腹部的摄像机它占据了一个专用的腹腔镜端口， 需要摄像机操作员操纵和导航摄像机以进行可视化。视野狭窄 并且难以推断场景的深度。为了便于手术，摄像机需要靠近 手术野，使其易于弄脏，雾化和飞溅，可能会模糊可视化，并需要 清洗.摄像机的位置也可能干扰手术器械的操作。 为了解决这些与当前腹腔镜可视化相关的缺陷，在本项目中，我们将 开发一种范式转换、集成、全景、灵活、沉浸式3D腹腔镜可视化系统 称为EasyVis，可以显著提高腹腔镜手术的效率。EasyVis直接 集成了多个微型摄像机及其外围设备，包括光源和微型投影仪， 手术端口。它提供了一个不间断的，腹腔内，灵活，身临其境的3D视图，从任意 虚拟视点和视角，以及任何特定区域的特写3D可视化，所有这些都在直接， 通过语音命令，外科医生无需手动，轻松全面地控制手术， 摄像头导航EasyVis不占用任何额外的手术端口， 弄脏、起雾和飞溅，以及干扰和堵塞器械。与EasyVis，外科医生 具有增强的可视化（质量更好，易于控制）和更少的腹腔镜中断 操作过程中的操作。 将追求三个具体目标。首先，我们将开发一个成像系统， 微型照相机阵列、光源和微型投影仪。该成像系统提供灵活的可选择的 具有自动对焦和缩放功能以及实时场景稳定功能的视点和视角。一 还将开发能够通过手术端口部署成像系统的系统。二是 开发算法，以实现灵活的全局和局部视图，以及沉浸式的任意视图3D可视化，所有 通过手术医生佩戴的人机界面。第三，我们将验证我们的腹腔镜 可视化系统与无生命和动物模型。

Abstract Laparoscopic or “minimally invasive” procedures are frequently performed in surgery today. It provides numerous clinical benefits to patients and has become the gold standard surgical procedure for many intraabdominal and thoracic procedures. Existing laparoscopes provide visualization of the operative field through a video camera inserted into the patient’s abdomen. It occupies a dedicated laparoscopic port and requires a camera operator to maneuver and navigate the camera for visualization. The field of view is narrow and the depth of the scene is difficult to infer. To facilitate surgery, the camera needs to be in proximity to the operative field, making it prone to smudging, fogging, and splatter that may obscure visualization and require cleaning. The position of the camera may also interferes with the operating surgical instruments. To address these deficiencies associated with current laparoscopic visualization, in this project, we will develop a paradigm-shift, integrated, panoramic, flexible, immersive 3D laparoscopic visualization system called EasyVis that could significantly improve the efficiency of laparoscopic surgery. EasyVis directly integrates multiple microcameras and their peripherals including light sources and miniaturized projectors with the surgical ports. It provides an uninterrupted, intra-abdominal, flexible, and immersive 3D view from arbitrary virtual viewpoints and viewing angles, and close-up 3D visualization of any specific area, all under direct, hands-free, easy and full control of the operating surgeon through voice commands, eliminating the need of camera navigation. EasyVis does not occupy any extra surgical port and solves the problems associated with smudging, fogging and splatter, and interference with and occlusion of instruments. With EasyVis, the surgeon is provided with enhanced visualization (better quality, easy control) and less interruption due to laparoscope manipulation during the operation. Three specific aims will be pursued. First, we will develop an imaging system including heterogeneous microcamera arrays, light sources, and miniature projectors. This imaging system provides flexibly selectable viewpoints and viewing angles with auto-focus and zooming capabilities and real-time scene stabilization. A system that can deploy the imaging system through a surgical port will also be developed. Second, we will develop algorithms to achieve flexible global and local views, and immersive, any-view 3D visualization, all through a human-computer interface worn by the operating surgeon. Third, we will validate our laparoscopic visualization system with inanimate and animal models.