2D-3D Reconstruction for internal organs using Deep Learning Techniques

点击翻译按钮获取中文摘要

The purpose of this research is to develop a data-driven approach which can reconstruct 3D data from 2D images. A typical application is to reconstruct 3D CT data from x-ray images. If we can realize this goal, we can locate the tumors much more accurately than the usual case during a surgery.To fulfill the goal, we have proposed two different approaches for the 2D-3D reconstruction. First, we reconstructed the 3D CT data from fewer views than normal cases and then reduced the number of views. From-less-to-more reconstruction was implemented in Z axis, and the results were published recently. We tried to reduce the number of views to approximate 2D. However, the results were not so good.Therefore, we proposed the second approach where we implemented an existing neural network X2CT-GAN. This model can reconstruct 3D CT data from two orthogonal X-ray images. However, the accuracy is not good enough (SSIM is about 0.62). Although we used data augmentation to improve its performance, the SSIM is about 0.74, which is still too low for clinic applications. This time, we exploited the 3D topology of the training CT data, and added the 3D topology features to the X2CT-GAN network. In our model, the system can reconstruct 3D CT data from two x-ray images which are not necessarily the orthogonal ones.We compared our approach with the original X2CT-GAN, and found our method achieved better performance. Currently, we are implementing more comparison experiments to demonstrate the effectiveness of the proposed model.