3D reconstruction of Borobudur reliefs from 2D monocular photographs based on soft-edge enhanced deep learning
3D reconstruction of Borobudur reliefs from 2D monocular photographs based on soft-edge enhanced deep learning
复制标题
基于软边缘增强深度学习的 2D 单目照片对婆罗浮屠浮雕的 3D 重建
DOI：
10.1016/j.isprsjprs.2021.11.007
复制
发表时间：
2022
影响因子：
12.7
通讯作者：
Tanaka Satoshi
中科院分区：
文献类型：
--
作者：
Pan Jiao;Li Liang;Yamaguchi Hiroshi;Hasegawa Kyoko;Thufail Fadjar I.;Brahmantara;Tanaka Satoshi
提出了一种基于深度学习的三维(3D)文化遗产对象重建方法，该方法针对三维文化遗产对象被破坏或消失而无法进行3D扫描来创建数字档案。该重建方法使用单目照片，并基于单目深度估计网络来预测其深度值。虽然最近的研究主要集中在具有尖锐(高曲率)边缘和大深度差的场景，如室内和室外场景，但所提出的方法旨在发挥其在小深度差和软(低曲率)边缘的场景中的性能。其核心思想是将软边缘作为深度估计网络中的引导信息。我们通过将其应用于联合国教科文组织世界遗产婆罗浮图神庙的隐蔽浮雕，证明了建议方法的有效性。隐藏的浮雕被埋在石墙后面，不再可见。浮雕的3D信息只能从它们以前的单目照片中估计出来。对验证数据的重建准确率达到97%。我们还将重建的隐形地貌与可视区域的3D扫描数据进行了整合，并对整合后的数据进行了透视可视化。这种可视化可以同时向我们展示寺庙的可见区域和不可见区域。
We propose a deep learning-based reconstruction method for three-dimensional (3D) cultural heritage objects, which are destroyed or disappear and thus are unavailable for 3D scanning to create digital archives. The proposed reconstruction method uses a single monocular photo and predicts its depth values based on a monocular depth estimation network. While most recent studies have focused on scenes with sharp (high-curvature) edges and large depth differences, such as indoor and outdoor scenes, the proposed method is designed to exert its performance on scenes with small depth differences and soft (low-curvature) edges. The key idea is to involve the soft edges as guidance information in the depth estimation network. We demonstrate the effectiveness of the proposed method by applying it to the hidden reliefs of the Borobudur temple, a UNESCO world heritage site. The hidden reliefs were buried behind stone walls and are no longer visible. The 3D information of the reliefs can only be estimated from their old monocular photos. We achieved 97% reconstruction accuracy for validation data. We also integrate the reconstructed hidden reliefs with 3D scanned data of the visible regions and execute see-through visualization for the integrated data. This visualization can simultaneously show us both visible regions and invisible regions of the temple.
尼泊尔低成本图像遗产记录的可行性研究
DOI：
--
发表时间：
2017
期刊：
影响因子：
--
作者：
H. K. Dhonju;W. Xiao;V. Sarhosis;J. Mills;S. Wilkinson;Zheng Wang;L. Thapa;U. S. Panday
通讯作者：
U. S. Panday