权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

2D-3D Reconstruction for internal organs using Deep Learning Techniques

使用深度学习技术进行内脏器官 2D-3D 重建

基本信息

批准号：
20K20167
负责人：
武淑瓊
金额：
$ 2.75万
依托单位：
Osaka University
依托单位国家：
日本
项目类别：
Grant-in-Aid for Early-Career Scientists
财政年份：
2020
资助国家：
日本
起止时间：
2020-04-01 至 2024-03-31
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-20K20167/
关键词：
3D reconstruction Deep learning GAN Deformation estimation 2D-3D registration 3D organ reconstruction deep Learning 2D-3D reconstruction CT images deformation estimation radiotherapy

项目摘要

The purpose of this research is to develop a data-driven approach which can reconstruct 3D data from 2D images. A typical application is to reconstruct 3D CT data from x-ray images. If we can realize this goal, we can locate the tumors much more accurately than the usual case during a surgery.To fulfill the goal, we have proposed two different approaches for the 2D-3D reconstruction. First, we reconstructed the 3D CT data from fewer views than normal cases and then reduced the number of views. From-less-to-more reconstruction was implemented in Z axis, and the results were published recently. We tried to reduce the number of views to approximate 2D. However, the results were not so good.Therefore, we proposed the second approach where we implemented an existing neural network X2CT-GAN. This model can reconstruct 3D CT data from two orthogonal X-ray images. However, the accuracy is not good enough (SSIM is about 0.62). Although we used data augmentation to improve its performance, the SSIM is about 0.74, which is still too low for clinic applications. This time, we exploited the 3D topology of the training CT data, and added the 3D topology features to the X2CT-GAN network. In our model, the system can reconstruct 3D CT data from two x-ray images which are not necessarily the orthogonal ones.We compared our approach with the original X2CT-GAN, and found our method achieved better performance. Currently, we are implementing more comparison experiments to demonstrate the effectiveness of the proposed model.

本研究的目的是开发一种数据驱动的方法，可以从二维图像中重建三维数据。一个典型的应用是从x射线图像重建三维CT数据。如果我们能实现这个目标，我们就能在手术中比通常情况下更准确地定位肿瘤。为了实现这一目标，我们提出了两种不同的2D-3D重建方法。首先，我们用比正常病例更少的视图重建三维CT数据，然后减少视图数。在Z轴上进行了从少到多的重构，并于近期发表了结果。我们试图将视图数量减少到接近2D。然而，结果并不是那么好。因此，我们提出了第二种方法，其中我们实现了现有的神经网络X2CT-GAN。该模型可以从两个正交的x射线图像中重建三维CT数据。然而，准确度不够好（SSIM约为0.62）。虽然我们使用了数据增强来提高其性能，但SSIM约为0.74，对于临床应用来说仍然太低。这一次，我们利用训练CT数据的三维拓扑，并将三维拓扑特征添加到X2CT-GAN网络中。在我们的模型中，系统可以从两张不一定正交的x射线图像中重建三维CT数据。我们将我们的方法与原始的X2CT-GAN进行了比较，发现我们的方法取得了更好的性能。目前，我们正在进行更多的对比实验来证明所提出模型的有效性。