3DensiNet: A Robust Neural Network Architecture towards 3D Volumetric Object Prediction from 2D Image
3DensiNet: A Robust Neural Network Architecture towards 3D Volumetric Object Prediction from 2D Image
复制标题
DOI：
10.1145/3123266.3123340
复制
发表时间：
2017-10
期刊：
影响因子：
--
通讯作者：
M. Wang;Lingjing Wang;Yi Fang
中科院分区：
文献类型：
--
作者：
M. Wang;Lingjing Wang;Yi Fang
在三维视觉计算中，从单个二维图像生成/预测三维体对象是一个相当具有挑战性但有意义的任务。在本文中，我们提出了一种新的神经网络架构，命名为“3DensiNet”，它使用密度热图作为2D到3D转换的中间监督工具。具体来说，我们首先提出了一个2D密度热图到3D体积对象编码-解码网络，它优于经典的3D自动编码器。然后，我们表明，使用2D图像预测其密度热图通过2D到2D的编码-解码网络是可行的。此外，我们利用对抗性损失来微调我们的网络，这将改进生成/预测的3D体素对象，使其与地面真实体素对象更相似。从2D图像进行3D体积预测的实验结果表明，3DensiNet在处理从单个2D图像生成/预测3D体积对象方面优于其他最先进的技术的上级性能。
3D volumetric object generation/prediction from single 2D image is a quite challenging but meaningful task in 3D visual computing. In this paper, we propose a novel neural network architecture, named "3DensiNet", which uses density heat-map as an intermediate supervision tool for 2D-to-3D transformation. Specifically, we firstly present a 2D density heat-map to 3D volumetric object encoding-decoding network, which outperforms classical 3D autoencoder. Then we show that using 2D image to predict its density heat-map via a 2D to 2D encoding-decoding network is feasible. In addition, we leverage adversarial loss to fine tune our network, which improves the generated/predicted 3D voxel objects to be more similar to the ground truth voxel object. Experimental results on 3D volumetric prediction from 2D images demonstrates superior performance of 3DensiNet over other state-of-the-art techniques in handling 3D volumetric object generation/prediction from single 2D image.