Development of endoscope navigation system based on 3D image recognition, understanding, and video image analysis

本研究项目的目的是开发一种灵活的内窥镜导航系统，该系统通过显示从每个患者的孔术三维医学图像中获得的导航信息来指导医生。该系统是一个真实与虚拟的内窥镜系统。我们已经开发了柔性内窥镜导航系统所需的以下基本技术：(1)柔性内窥镜跟踪，(2)显示当前观察壁外解剖结构所需的技术，(3)辅助柔性内窥镜检查和手术的方法。(1)柔性内窥镜跟踪我们开发了几种方法，可以从真实的内窥镜视频和术前CT扫描中估计柔性内窥镜摄像机的运动。本文研究了三种跟踪方法：(a)利用极极几何分析从视频序列中直接估计摄像机运动；(b)利用真实和虚拟内窥镜图像之间逐帧图像配准（一种2D/3D图像配准）估计摄像机运动；(c)利用卡尔曼滤波预测摄像机运动及其减少计算时间的应用。(2)当前观察器官壁外的解剖结构显示在内镜导航系统中，显示当前观察器官壁外存在的解剖结构非常重要。为了实现这一功能，我们开发了几种从输入CT图像中分割每个器官的方法。这些分割结果通过在内镜检查期间使用基于快速软件的体绘制可视化。(3)辅助内镜的导航信息我们研究了三种显示导航信息作为内镜辅助的方法：(a)支气管树的自动解剖标记，(b)内镜检查过程中未观察区域的检测，以及(c)使用图像变形显示信息。

The purpose of this research project is to develop a flexible endoscope navigation system, which guides a physician by presenting navigation information derived from pore-operative 3D medical images of each patient. The system is implemented as a fusion of real and virtual endoscopy system. We have developed the following elementary techniques that are required in the flexible endoscope navigation system : (1) flexible endoscope tracking, (2) techniques that are required for presenting anatomical structures beyond the wall currently observed, (3) methods for assisting flexible endoscopic examination and surgery.(1) Flexible endoscope trackingWe developed several methods for estimating flexible endoscopic camera motion from real endoscopic videos and preoperative CT scans. Here, three types of tracking methods were investigated : (a) direct camera motion estimation from video sequences by using epipolar geometry analysis, (b) camera motion estimation by frame-by-frame image registration between real and virtual endoscopic images (a kind of 2D/3D image registration), (c) camera motion prediction by Kalman filtering and its application for reducing computation time.(2) Anatomical structure display beyond the organ's wall being currendy observedIn the endoscopic navigation system, it is very important to display anatomical structure existing beyond the organ's wall being currently observed. For realizing this function, we developed several methods for segmenting each organ from input CT images. These segmentation results were visualized by using fast software-based volume rendering during endoscopic examinations.(3) Navigation information for assisting endoscopyWe investigated three methods for presenting navigation information as assistance of endoscopy : (a) automated anatomical labeling of bronchial trees, (b) detection of unobserved regions during endoscopy, and (c) information display using image deformation.