Space-time completion of video
Space-time completion of video
复制标题
DOI：
10.1109/tpami.2007.60
复制
发表时间：
2007-03-01
影响因子：
23.6
通讯作者：
Irani, Michal
中科院分区：
文献类型：
--
作者：
Wexler, Yonatan;Shechtman, Eli;Irani, Michal
本文提出了一种基于局部结构的缺失信息补全的新框架。它将完成任务作为一个具有明确定义的目标函数的全局优化问题，并推导出一种新的算法来优化它。相对于参考示例，缺失值被限制为形成连贯的结构。我们将该方法应用于复杂动态场景视频序列中大时空“洞”的时空补全。通过从视频的可用部分采样时空补丁来填充缺失的部分，同时强制孔内和周围的所有补丁之间的全局时空一致性。静态场景部分与动态行为同时完成，可以产生逼真的视频序列和图像。时空视频补全可用于多种任务，包括但不限于：1）通过补全适当的静态或动态背景信息来复杂地去除（不需要的静态或动态对象）视频。 2）修正老电影中丢失/损坏的视频帧。 3) 通过替换不需要的元素来修改视觉故事。 4) 通过扩展较小的纹理来创建视频纹理。 5) 创建完整的视场稳定视频。 6）由于图像是单帧视频，我们也将该方法应用于这种特殊情况。
This paper presents a new framework for the completion of missing information based on local structures. It poses the task of completion as a global optimization problem with a well-defined objective function and derives a new algorithm to optimize it. Missing values are constrained to form coherent structures with respect to reference examples. We apply this method to space-time completion of large space-time "holes" in video sequences of complex dynamic scenes. The missing portions are filled in by sampling spatio-temporal patches from the available parts of the video, while enforcing global spatio-temporal consistency between all patches in and around the hole. The consistent completion of static scene parts simultaneously with dynamic behaviors leads to realistic looking video sequences and images. Space-time video completion is useful for a variety of tasks, including, but not limited to: 1) Sophisticated video removal (of undesired static or dynamic objects) by completing the appropriate static or dynamic background information. 2) Correction of missing/corrupted video frames in old movies. 3) Modifying a visual story by replacing unwanted elements. 4) Creation of video textures by extending smaller ones. 5) Creation of complete field-of-view stabilized video. 6) As images are one-frame videos, we apply the method to this special case as well.