Optical Flow Regularization of Implicit Neural Representations for Video Frame Interpolation
Optical Flow Regularization of Implicit Neural Representations for Video Frame Interpolation
复制标题
DOI：
10.48550/arxiv.2206.10886
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Weihao Zhuang;T. Hascoet;R. Takashima;T. Takiguchi
中科院分区：
文献类型：
--
作者：
Weihao Zhuang;T. Hascoet;R. Takashima;T. Takiguchi
最近的工作表明，隐式神经表征（INR）的能力进行有意义的表示信号的衍生物。在这项工作中，我们利用这个属性来执行视频帧插值（VFI），明确约束的INR的导数，以满足光流约束方程。我们只使用目标视频及其光流在有限的运动范围内实现了最先进的VFI，而无需从额外的训练数据中学习插值算子。我们进一步表明，约束INR导数不仅可以更好地插值中间帧，而且还可以提高窄网络拟合观察到的帧的能力，这表明了视频压缩和INR优化的潜在应用。
Recent works have shown the ability of Implicit Neural Representations (INR) to carry meaningful representations of signal derivatives. In this work, we leverage this property to perform Video Frame Interpolation (VFI) by explicitly constraining the derivatives of the INR to satisfy the optical flow constraint equation. We achieve state of the art VFI on limited motion ranges using only a target video and its optical flow, without learning the interpolation operator from additional training data. We further show that constraining the INR derivatives not only allows to better interpolate intermediate frames but also improves the ability of narrow networks to fit the observed frames, which suggests potential applications to video compression and INR optimization.