EasyVis: Flexible, immersive three-dimensional laparoscopic surgical visualization through multi-camera arrays

摘要 如今，在外科手术中，腹腔镜术或“微创”手术很常见。它提供了 给患者带来了许多临床益处，并已成为许多人的黄金标准外科手术 腹内和胸部手术。现有的腹腔镜提供了手术视野的可视化。 通过插入病人腹部的摄像机。它占用了一个专用的腹腔镜口 需要相机操作员操纵和导航相机以实现可视化。视场很窄 而且现场的深度也很难推断。为了便于手术，摄像机需要靠近 作业场，容易产生污迹、雾化和飞溅，这可能会影响视觉效果，并需要 打扫卫生。摄像机的位置也可能会干扰手术器械。 为了解决与当前的腹腔镜可视化相关的这些不足，在这个项目中，我们将 开发一种范式转换、集成、全景、灵活、沉浸式的3D腹腔镜可视化系统 这项名为EasyVis的技术可以显著提高腹腔镜手术的效率。EasyVis直接 将多个微型摄像机及其外围设备(包括光源和微型投影仪)与 外科手术口子。它提供不间断、腹内、灵活、身临其境的任意3D视图 虚拟视点和视角，以及任意特定区域的特写3D可视化，所有这些都是在Direct、 通过语音命令对手术医生进行免提、轻松和完全控制，无需 相机导航。EasyVis不占用任何额外的手术端口，并解决了与 涂抹、雾化和飞溅，以及干扰和遮挡仪器。和EasyVis，外科医生 增强了可视化(质量更好，易于控制)，并减少了由于腹腔镜而造成的中断 在手术过程中的操纵。 将追求三个具体目标。首先，我们将开发一种包括异构性的成像系统 微型相机阵列、光源和微型投影仪。该成像系统提供灵活的可选 具有自动对焦和缩放功能以及实时场景稳定的视点和视角。一个 还将开发可以通过外科端口部署成像系统的系统。第二，我们将 开发算法以实现灵活的全局和局部视图，以及身临其境的任意视图3D可视化，所有这些 通过手术医生佩戴的人机界面。第三，我们将验证我们的腹腔镜手术 具有无生命和动物模型的可视化系统。

Abstract Laparoscopic or “minimally invasive” procedures are frequently performed in surgery today. It provides numerous clinical benefits to patients and has become the gold standard surgical procedure for many intraabdominal and thoracic procedures. Existing laparoscopes provide visualization of the operative field through a video camera inserted into the patient’s abdomen. It occupies a dedicated laparoscopic port and requires a camera operator to maneuver and navigate the camera for visualization. The field of view is narrow and the depth of the scene is difficult to infer. To facilitate surgery, the camera needs to be in proximity to the operative field, making it prone to smudging, fogging, and splatter that may obscure visualization and require cleaning. The position of the camera may also interferes with the operating surgical instruments. To address these deficiencies associated with current laparoscopic visualization, in this project, we will develop a paradigm-shift, integrated, panoramic, flexible, immersive 3D laparoscopic visualization system called EasyVis that could significantly improve the efficiency of laparoscopic surgery. EasyVis directly integrates multiple microcameras and their peripherals including light sources and miniaturized projectors with the surgical ports. It provides an uninterrupted, intra-abdominal, flexible, and immersive 3D view from arbitrary virtual viewpoints and viewing angles, and close-up 3D visualization of any specific area, all under direct, hands-free, easy and full control of the operating surgeon through voice commands, eliminating the need of camera navigation. EasyVis does not occupy any extra surgical port and solves the problems associated with smudging, fogging and splatter, and interference with and occlusion of instruments. With EasyVis, the surgeon is provided with enhanced visualization (better quality, easy control) and less interruption due to laparoscope manipulation during the operation. Three specific aims will be pursued. First, we will develop an imaging system including heterogeneous microcamera arrays, light sources, and miniature projectors. This imaging system provides flexibly selectable viewpoints and viewing angles with auto-focus and zooming capabilities and real-time scene stabilization. A system that can deploy the imaging system through a surgical port will also be developed. Second, we will develop algorithms to achieve flexible global and local views, and immersive, any-view 3D visualization, all through a human-computer interface worn by the operating surgeon. Third, we will validate our laparoscopic visualization system with inanimate and animal models.