Keyframe extraction using Pearson correlation coefficient and color moments
Keyframe extraction using Pearson correlation coefficient and color moments
复制标题
DOI：
10.1007/s00530-019-00642-8
复制
发表时间：
2020-06-01
影响因子：
3.9
通讯作者：
Khare, Ashish
中科院分区：
文献类型：
--
作者：
Bommisetty, Reddy Mounika;Prakash, Om;Khare, Ashish
关键帧提取在视频摘要、视频管理和检索等实时视频处理应用中起着重要的作用。关键帧捕获了其镜头的全部内容，并且不包含任何冗余信息。由于不同类别的视频具有不同的视觉特征，因此关键帧提取算法面临着挑战。因此，单一特征不足以捕捉多种视频的视觉特征。为了解决这个问题，我们提出了一种方法的关键帧提取，使用杂交的功能。在本文中，我们提出了一种新的基于镜头检测的关键帧提取算法的基础上结合两个特征：一个是皮尔逊相关系数（PCC），另一个是颜色矩（CM）。PCC的线性变换不变性的性质，使所提出的算法，以及在不同的光照条件下工作。另一方面，颜色矩的尺度和旋转不变性属性对于可能以不同姿势和方向存在的复杂对象的表示是有益的。这些持续的原因支持这两个特征的结合，这带来了显着的好处，关键帧提取的方法。该方法利用组合特征集（PCC和CM）检测镜头边界。从每个镜头中，选择具有最高平均值和标准差的帧作为关键帧。此外，另一个重要的贡献是，我们通过收集不同类别的视频（如电影，新闻，连续剧，动画和个人访谈）开发了一个新的数据集，并将其在线提供。该方法在三个数据集上进行了实验：两个公开的数据集和一个我们开发的数据集。所提出的方法对这些数据集的性能进行了评估的基础上不同的评价参数：品质因数，检测百分比，准确性和缺失因子。所提出的工作的主要优点在于，它是能够检测突然和逐渐拍摄过渡的事实。在实时视频中，通常会出现突然和小的过渡。实验结果表明，该方法的上级性能优于其他国家的最先进的方法。
Keyframe extraction plays a significant role in wide variety of real-time video processing applications such as video summarization, video management and retrieval, etc. A keyframe captures the whole content of its shot and does not contain any redundant information. The keyframe extraction algorithms are facing challenges due to different visual characteristics in videos of different categories. Therefore, a single feature is not enough to capture visual characteristics of a variety of videos. In order to tackle this problem, we propose an approach of keyframe extraction that uses hybridization of features. In the present article, we propose a novel shot detection-based keyframe extraction algorithm based on combination of two features: one is Pearson correlation coefficient (PCC) and other is color moments (CM). The linear transformation invariance property of PCC facilitates the proposed algorithm to work well under varying lighting conditions. On the other hand, the scale and rotation invariance properties of color moments are beneficial for representation of complex objects that may be present in different poses and orientations. These sustained reasons support the combination of these two features, which brings significant benefits for keyframe extraction in the proposed method. The proposed method detects shot boundaries by employing combo feature set (PCC and CM). From each shot, the frame with highest mean and standard deviation is selected as keyframe. Furthermore, another important contribution is that we developed a new dataset by collecting the videos of different categories such as movies, news, serials, animations and personal interviews and made it available online. The proposed method is experimented on three datasets: two publicly available datasets and one dataset developed by us. The performance of the proposed method on these datasets has been evaluated on the basis of different evaluation parameters: figure of merit, detection percentage, accuracy, and missing factor. Principal advantage of proposed work lies in the fact that it is capable to detect both the abrupt and gradual shot transitions. In real-time videos, it is common to have abrupt and small transitions. The experimental results show the superior performance of the proposed method over the other state-of-the-art methods.