VC-Net: Deep Volume-Composition Networks for Segmentation and Visualization of Highly Sparse and Noisy Image Data
VC-Net: Deep Volume-Composition Networks for Segmentation and Visualization of Highly Sparse and Noisy Image Data
复制标题
DOI：
10.1109/tvcg.2020.3030374
复制
发表时间：
2021-02-01
影响因子：
5.2
通讯作者：
Zhong, Zichun
中科院分区：
文献类型：
--
作者：
Wang, Yifan;Yan, Guoli;Zhong, Zichun
这项工作的基本动机是提出一种新的可视化引导的计算范例，将直接的三维体处理和体绘制线索相结合，以实现有效的三维探索。例如，体内微结构的提取和可视化一直是一个具有挑战性的问题。然而，由于脑血管数据的高度稀疏性和噪声，以及微血管的高度复杂的几何和拓扑变化，提取完整的三维血管结构并以高保真的三维方式可视化仍然是非常具有挑战性的。本文提出了一种端到端的深度学习方法VC-Net，通过将最大强度投影(MIP)生成的图像成分嵌入到3D体积图像学习过程中，以提高整体性能，从而稳健地提取3D微血管结构。其核心新颖性是自动利用体可视化技术(例如，MIP-一种三维体图像的体绘制方案)来增强深度学习级别的3D数据探索。MIP嵌入特征可以增强局部血管信号(通过抵消噪声)，并适应血管的几何变异性和可扩展性，这在微血管跟踪中具有重要意义。提出了一种多流卷积神经网络(CNN)框架，通过将二维特征向量反投影到三维体嵌入空间中，分别有效地学习三维体和二维MIP特征向量，并在联合体组成嵌入空间中探索它们之间的相互依赖关系。值得注意的是，提出的框架可以更好地捕获小微血管，提高船只的连通性。据我们所知，这是第一次提出一个深度学习框架来构造联合卷积嵌入空间，在这个空间中，基于体绘制的二维投影和三维体绘制计算的血管概率可以协同探索和集成。使用大量的公共和真实患者(微)脑血管图像数据，对实验结果进行了评估，并与传统的3D血管分割方法和深度学习的最新进展进行了比较。应用该方法对稀疏和复杂的三维微血管结构进行了精确的分割和可视化，显示了该方法在血管疾病的强大的MR动脉和静脉造影诊断中的潜力。
The fundamental motivation of the proposed work is to present a new visualization-guided computing paradigm to combine direct 3D volume processing and volume rendered clues for effective 3D exploration. For example, extracting and visualizing microstructures in-vivo have been a long-standing challenging problem. However, due to the high sparseness and noisiness in cerebrovasculature data as well as highly complex geometry and topology variations of micro vessels, it is still extremely challenging to extract the complete 3D vessel structure and visualize it in 3D with high fidelity. In this paper, we present an end-to-end deep learning method, VC-Net, for robust extraction of 3D microvascular structure through embedding the image composition, generated by maximum intensity projection (MIP), into the 3D volumetric image learning process to enhance the overall performance. The core novelty is to automatically leverage the volume visualization technique (e.g., MIP - a volume rendering scheme for 3D volume images) to enhance the 3D data exploration at the deep learning level. The MIP embedding features can enhance the local vessel signal (through canceling out the noise) and adapt to the geometric variability and scalability of vessels, which is of great importance in microvascular tracking. A multi-stream convolutional neural network (CNN) framework is proposed to effectively learn the 3D volume and 2D MIP feature vectors, respectively, and then explore their inter-dependencies in a joint volume-composition embedding space by unprojecting the 2D feature vectors into the 3D volume embedding space. It is noted that the proposed framework can better capture the small/micro vessels and improve the vessel connectivity. To our knowledge, this is the first time that a deep learning framework is proposed to construct a joint convolutional embedding space, where the computed vessel probabilities from volume rendering based 2D projection and 3D volume can be explored and integrated synergistically. Experimental results are evaluated and compared with the traditional 3D vessel segmentation methods and the state-of-the-art in deep learning, by using extensive public and real patient (micro- )cerebrovascular image datasets. The application of this accurate segmentation and visualization of sparse and complicated 3D microvascular structure facilitated by our method demonstrates the potential in a powerful MR arteriogram and venogram diagnosis of vascular disease.