Deep learning model to reconstruct 3D cityscapes by generating depth maps from omnidirectional images and its application to visual preference prediction
Deep learning model to reconstruct 3D cityscapes by generating depth maps from omnidirectional images and its application to visual preference prediction
复制标题
DOI：
10.1017/dsj.2020.27
复制
发表时间：
2020-11
期刊：
影响因子：
2.4
通讯作者：
A. Takizawa;Hina Kinugawa
中科院分区：
文献类型：
--
作者：
A. Takizawa;Hina Kinugawa
摘要我们开发了一种从相应的城市景观全向图像生成全向深度图的方法，方法是使用Pix2pix学习由计算机图形学生成的每一对全向和深度图。在不同场地和天空条件下拍摄的不同系列图像的训练模型被应用于街景图像，以生成深度图。然后对生成的深度图的有效性进行了定量和直观的评估。此外，我们还利用多名参与者对谷歌街景图片进行了评估。对于一般矩形图像和全方位图像，我们使用深度卷积神经网络的分类方法构建了一个模型来预测这些图像在有和没有生成深度图的情况下的偏好标签。结果表明，城市景观偏好预测模型的泛化性能在多大程度上取决于卷积模型的类型和生成的深度图的存在或不存在。
Abstract We developed a method to generate omnidirectional depth maps from corresponding omnidirectional images of cityscapes by learning each pair of an omnidirectional and a depth map, created by computer graphics, using pix2pix. Models trained with different series of images, shot under different site and sky conditions, were applied to street view images to generate depth maps. The validity of the generated depth maps was then evaluated quantitatively and visually. In addition, we conducted experiments to evaluate Google Street View images using multiple participants. We constructed a model that predicts the preference label of these images with and without the generated depth maps using the classification method with deep convolutional neural networks for general rectangular images and omnidirectional images. The results demonstrate the extent to which the generalization performance of the cityscape preference prediction model changes depending on the type of convolutional models and the presence or absence of generated depth maps.