DCVGAN: Depth Conditional Video Generation
DCVGAN: Depth Conditional Video Generation
复制标题
DOI：
10.1109/icip.2019.8803764
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Yuki Nakahira;K. Kawamoto
中科院分区：
文献类型：
--
作者：
Yuki Nakahira;K. Kawamoto
在过去的几年里，已经提出了几种用于视频生成的生成对抗网络（GANs），但它们大多只使用彩色视频来训练生成模型。然而，为了使模型更准确地理解场景动态，除了光学信息外，三维几何信息也很重要。本文将深度视频和彩色视频结合起来，提出了一种用于视频生成的GAN结构。在我们架构的生成器中，在前半部分生成深度视频，后半部分通过求解深度到颜色的域转换生成彩色视频。通过对场景动态建模，重点关注深度信息，我们能够制作出比传统方法更高质量的视频。此外，我们表明，当评估面部表情和手势数据集时，我们的方法在种类和质量方面都比传统方法产生更好的视频样本。代码和生成的示例视频在Github1上公开提供。
In the past few years, several generative adversarial networks (GANs) for video generation have been proposed although most of them only use color videos to train the generative model. However, to make the model understand scene dynamics more accurately, not only optical information but also three-dimensional geometrical information is important. In this paper, using depth video together with color video, we propose a GAN architecture for video generation. In the generator of our architecture, the depth video is generated in the first half and in the second half, the color video is generated by solving the domain translation from the depth to the color. By modeling the scene dynamics with a focus on the depth information, we were able to produce videos of higher quality than the conventional method. Furthermore, we show that our method produces better video samples than ones by conventional method in terms of both variety and quality when evaluating on facial expression and hand gesture datasets. The codes and generated sample videos are publicly available on Github1.