Learning Disentangled Representations of Video with Missing Data
Learning Disentangled Representations of Video with Missing Data
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Armand Comas Massague;Chi Zhang;Z. Feric;O. Camps;Rose Yu
中科院分区：
文献类型：
--
作者：
Armand Comas Massague;Chi Zhang;Z. Feric;O. Camps;Rose Yu
在学习视频序列的表示时，缺失的数据提出了重大挑战。我们提出了解纠缠输入视频自动编码器（DIVE），这是一种深度生成模型，可以在存在缺失数据的情况下输入和预测未来的视频帧。具体来说，DIVE引入了一个缺失潜在变量，将隐藏的视频表示分解为每个物体的静态和动态外观、姿态和缺失因素，同时它将数据缺失的每个物体轨迹归因。在移动的MNIST数据集上，有各种缺失的场景，DIVE的性能大大优于最先进的基线。我们还对真实世界的motchallenge行人数据集进行了比较，这证明了我们的方法在更现实的环境中的实用价值。
Missing data poses significant challenges while learning representations of video sequences. We present Disentangled Imputed Video autoEncoder (DIVE), a deep generative model that imputes and predicts future video frames in the presence of missing data. Specifically, DIVE introduces a missingness latent variable, disentangles the hidden video representations into static and dynamic appearance, pose, and missingness factors for each object, while it imputes each object trajectory where data is missing. On a moving MNIST dataset with various missing scenarios, DIVE outperforms the state of the art baselines by a substantial margin. We also present comparisons for real-world MOTSChallenge pedestrian dataset, which demonstrates the practical value of our method in a more realistic setting.