SURFGenerator: Generative Adversarial Network Modeling for Synthetic Flooding Video Generation
SURFGenerator: Generative Adversarial Network Modeling for Synthetic Flooding Video Generation
复制标题
DOI：
10.1109/ijcnn55064.2022.9891969
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Stephen Lamczyk;Kwame Ampofo;Behrouz Salashour;M. Cetin;K. Iftekharuddin
中科院分区：
文献类型：
--
作者：
Stephen Lamczyk;Kwame Ampofo;Behrouz Salashour;M. Cetin;K. Iftekharuddin
目前缺乏标记的大规模洪水视频数据集阻碍了洪水视频的语义分割工作。洪水视频语义分割在许多关键应用领域都是必需的，包括由于海平面上升和气候变化而导致的经常性洪水分析。为了解决缺乏标记的洪水视频，这项工作提出了一种新的基于生成对抗网络（GAN）的合成城市经常性洪水（SURF）生成器（SURFGenerator），用于生成不同水深水平的洪水合成视频。我们首先使用Blender [1]和Mantaflow [2]中基于物理的水模拟工具为我们的模型生成洪水信息合成渲染视频的大规模训练样本。然后开发了一个两部分模型，灵感来自以前的工作，由使用LinkNet [3]的语义分割（masker）网络和现成的vid2vid [4] painter网络组成。掩蔽推理网络使用渲染图像和相应的二进制分割掩码进行训练，以识别视频中特定深度级别的潜在泛洪区域。最后，使用分割的渲染视频作为输入单独训练画家网络。画家网络获取遮罩和初始视频，并在遮罩区域中生成合成水，以创建输出洪水视频。据我们所知，这项工作在文献中第一次生成了具有物理相关特征的合成洪水视频，例如当汽车在被洪水淹没的街道上移动时，移动的涟漪和小浪。这些合成生成的真实洪水视频可用于生成大规模标记图像，用于使用深度网络模型对洪水淹没的街道和其他城市环境进行语义分割和水深估计。我们在https://github.com/visionlabodu/SURFGenerator上免费提供我们的代码和数据。
The current lack of labeled large-scale flooding video datasets hinders work in semantic segmentation of flooding videos. Flooding video semantic segmentation is needed in many critical application areas, including recurrent flooding analysis due to sea-level rise and climate change. To address the lack of labeled flooding videos, this work proposes a novel generative adversarial network (GAN)-based Synthetic Urban Recurrent Flooding (SURF) Generator (SURFGenerator) for generating synthetic videos of flooding at different water depth levels. We first generate large-scale training samples of synthetically rendered videos of flooding information using physics-based water simulation tools within Blender [1] and Mantaflow [2] for our model. A two-part model is then developed, inspired by previous work, composed of a semantic segmentation (masker) network using LinkNet [3] followed by an off-the-shelf vid2vid [4] painter network. The masker inference network is trained with rendered images and corresponding binary segmentation masks to identify potential flooding areas for a specific depth level in the video. Finally, the painter network is trained separately using segmented render videos as input. The painter network takes the masks and initial video and generates synthetic water in the masked areas to create an output flooding video. To the best of our knowledge, for the first time in literature, this work generates synthetic flooding videos with physically relevant features such as moving ripples and small waves when cars move on the flooded streets. These synthetically generated realistic flooding videos may be used to generate large-scale labeled images for semantic segmentation and water depth estimation on flooded streets and other urban settings using deep network models. We make our code and data freely available at https://github.com/visionlabodu/SURFGenerator.