I-Corps: Virtual 3D reconstruction of hollow organs from white light endoscopy

I-Corps：通过白光内窥镜对中空器官进行虚拟 3D 重建

• 批准号: 1602118
• 负责人: Audrey Bowden
• 金额: $ 5万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-11-01 至 2016-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1602118&HistoricalAwards=false
• 关键词: Corps; Virtual; 3D; reconstruction; hollow

White light endoscope (WLE) is a medical device that allows physicians to conduct real-time video explorations of the interior of organs to detect disease or guide surgery. Unfortunately, the WLE video data are too cumbersome in their native form to review post-treatment; hence, the information is often reduced to handwritten notes or a few still-image frames for inclusion in medical records. The loss of this visually rich information limits the ability of WLE to inform clinical decisions about treatment and surgery. In particular, diseases like bladder cancer, which holds the unfortunate distinction as the 4th most common cancer in men and the highest treatment cost per patient-lifetime of all cancers, would benefit from novel ways to review WLE video data to facilitate early detection of tumors and to better track changes in the bladder wall of patients likely to experience recurrence ( 50%). The goal of this project is to develop new technology to produce 3D visualizations of the interior of hollow organs such as the bladder from WLE videos. The availability of such technology will provide physicians with new tools to make better informed decisions about treatment and surgery, as well as provide researchers with new technology to enable novel studies on disease progression, ultimately leading to better health outcomes and lower treatment costs for diseases like cancer.The proposed novel algorithm applies state-of-the-art techniques in computer vision to the problem of 3D reconstruction of the shape and surface appearance of hollow organs. A key innovation in the proposed approach is that the algorithm is suitable to reconstruct a full, 3D model of an organ from endoscopic video captured with standard clinical hardware and requires only minor modifications to the standard clinical workflow. The ability to create these reconstructions from standard equipment and workflows arises from careful design decisions regarding (1) the protocol for endoscopic video collection, (2) necessary image pre-processing steps and (3) the particular combination of state-of-the-art techniques developed in the computer vision community into an end-to-end pipeline unique for our application. In brief, the overall algorithm involves the following steps: down-sample raw image data, process selected frames, determine camera poses, extrapolate the organ surface as a mesh and apply image-based texture to the finalized mesh. The team's initial application is for reconstructing urinary bladder using standard rigid cystoscopy data; to date the team has validated the ability to perform 3D reconstructions using standard clinical cystoscopy videos of 30 human patients. As the proposed method can powerfully augment the visual medical record of internal organ appearance, it is broadly applicable to endoscopy and represents a significant advance in monitoring the appearance of a patient's organ over time, as may be well suited for applications such as cancer surveillance.

白光内窥镜（WLE）是一种医疗装置，允许医生对器官内部进行实时视频探索，以检测疾病或指导手术。不幸的是，WLE视频数据的本地形式太麻烦了，无法审查后处理。因此，这些信息通常被简化为手写笔记或一些静止图像的框架，以包含在病历中。视觉上丰富的信息的丢失限制了WLE为治疗和手术提供临床决策的能力。特别是，像膀胱癌这样的疾病将不幸的区别为男性中第四大癌症和所有癌症患者的最高治疗费用，将受益于所有癌症的新颖方法来审查WLE视频数据，以促进肿瘤的早期发现，并更好地跟踪患者的膀胱壁的变化（50％）。该项目的目的是开发新技术，以产生空心器官内部的3D可视化，例如WLE视频中的膀胱。这种技术的可用性将为医生提供新工具，以做出更好的治疗和手术决策，并为研究人员提供新技术，以实现有关疾病进展的新型研究，最终导致癌症等疾病的疾病和较低的治疗成本，例如癌症等疾病。拟议中的新型算法在计算机视觉中适用于3d Recormition和Shape of Shape and of Shape and of Shape and shape and shape and shape and shape and shape and shape and shape and shape and shape and shape and sype and comperstion的疾病。拟议方法中的一个关键创新是，该算法适合从具有标准临床硬件捕获的内窥镜视频中重建完整的3D器官3D模型，并且仅需要对标准临床工作流程进行较小的修改。从标准设备和工作流中创建这些重建的能力来自有关（1）内镜视频收集协议，（2）必要的图像预处理步骤，以及（3）计算机视觉社区中最先进技术的特定组合，成为我们应用程序独特的端点管道。简而言之，总体算法涉及以下步骤：下样本的原始图像数据，处理选定的帧，确定相机姿势，将器官表面推断为网格，然后将基于图像的纹理应用于最终的网格。该团队的初始应用是用于使用标准刚性膀胱镜检查数据重建膀胱；迄今为止，该团队已经验证了使用30名人类患者的标准临床镜检查视频进行3D重建的能力。由于所提出的方法可以有力地增强内脏器官外观的视觉病历，因此它广泛适用于内窥镜检查，并在监测患者器官随时间的外观方面具有重大进展，这可能非常适合于诸如癌症监测之类的应用。