Incorporating Event Information for High-Quality Video Interpolation
Incorporating Event Information for High-Quality Video Interpolation
复制标题
DOI：
10.1109/icnsc58704.2023.10319006
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Beibei Yang;Guangyu Jiang;Yan Fang;Weiling Li
中科院分区：
文献类型：
--
作者：
Beibei Yang;Guangyu Jiang;Yan Fang;Weiling Li
基于光流的视频插值是一种常用的方法，通过从现有中间帧生成新的中间帧来丰富视频细节。然而，它无法准确地再现不规则和快速移动物体的轨迹。为了实现高速场景的精确重建，在插值过程中融入事件信息是一种有效的方法。然而，现有方法不适合事件稀疏的场景。为了在这些场景下实现高质量的视频插值，本研究将事件信息纳入插值过程，采用三重思路：a）将移动目标视为前景并使用事件来描绘目标区域；b）根据事件信息将前景与背景进行匹配，以生成关键帧之间的时间帧；c）通过光流插值在关键帧和时间帧之间生成中间帧。实证研究表明，由于其有效地结合了事件信息，所提出的框架在生成用于视频插值的高质量帧方面优于最先进的方法。
Optical flow-based video interpolation is a commonly used method to enrich video details by generating new intermediate frames from existing ones. However, it cannot accurately reproduce the trajectories of irregular and fast-moving objects. To achieve the accurate reconstruction of high-speed scenes, incorporating event information during the interpolation process is an effective method. However, existing methods are not suitable for scenarios where events are sparse. To implement high-quality video interpolation for these scenarios, this study incorporates event information into the interpolation process with three-fold ideas: a) treating the moving target as the foreground and using events to delineate the target area, b) matching foreground to background based on the event information to generate temporal frame between keyframes, and c) generating intermediate frames between keyframe and temporal frame with optical flow interpolation. Empirical studies indicate that owing to its efficient incorporation of event information, the proposed framework outperforms state-of-the-art methods in generating high-quality frames for video interpolation.