Adaptive weighted fusion with new spatial and temporal fingerprints for improved video copy detection
Adaptive weighted fusion with new spatial and temporal fingerprints for improved video copy detection
复制标题
DOI：
10.1016/j.image.2014.05.002
复制
发表时间：
2014-08-01
影响因子：
3.5
通讯作者：
Ro, Yong Man
中科院分区：
文献类型：
--
作者：
Kim, Semin;Choi, Jae Young;Ro, Yong Man
在本文中，我们提出了一种新的和新颖的模态融合方法，旨在结合空间和时间的指纹信息，以提高视频拷贝检测性能。大多数先前开发的方法已经被限制为仅使用预先指定的权重来联合收割机组合空间和时间模态信息。因此，先前的方法可能无法自适应地调整取决于所比较的视频的时间方差之间的差异的时间指纹的重要性，从而导致视频拷贝检测中的性能降级。为了克服上述限制，所提出的方法已经被设计为提取两种类型的指纹信息：（1）由关键帧中的局部区域中的DCT系数的符号组成的空间指纹，以及（2）计算连续关键帧中的局部区域中的时间方差的时间指纹。此外，所谓的时间强度测量技术的开发，以定量表示的时间方差的量，它可以自适应地用来考虑的时间指纹比较的意义。实验结果表明，本文提出的模态融合方法在视频拷贝检测方面优于其他最先进的融合方法和流行的时空指纹。此外，所提出的方法可以节省39.0%，25.1%和46.1%的时间复杂度进行视频指纹匹配，而不会显着损失的检测精度为我们的合成数据集，TRECVID 2009 CCD任务，和MUSCLE-VCD 2007，分别。这一结果表明，我们提出的方法可以很容易地纳入到现实生活中的视频拷贝检测系统。(C)2014爱思唯尔有限公司版权所有。
In this paper, we propose a new and novel modality fusion method designed for combining spatial and temporal fingerprint information to improve video copy detection performance. Most of the previously developed methods have been limited to use only pre-specified weights to combine spatial and temporal modality information. Hence, previous approaches may not adaptively adjust the significance of the temporal fingerprints that depends on the difference between the temporal variances of compared videos, leading to performance degradation in video copy detection. To overcome the aforementioned limitation, the proposed method has been devised to extract two types of fingerprint information: (1) spatial fingerprint that consists of the signs of DCT coefficients in local areas in a keyframe and (2) temporal fingerprint that computes the temporal variances in local areas in consecutive keyframes. In addition, the so-called temporal strength measurement technique is developed to quantitatively represent the amount of the temporal variances; it can be adaptively used to consider the significance of compared temporal fingerprints. The experimental results show that the proposed modality fusion method outperforms other state-of-the-arts fusion methods and popular spatio-temporal fingerprints in terms of video copy detection. Furthermore, the proposed method can save 39.0%, 25.1%, and 46.1% time complexities needed to perform video fingerprint matching without a significant loss of detection accuracy for our synthetic dataset, TRECVID 2009 CCD Task, and MUSCLE-VCD 2007, respectively. This result indicates that our proposed method can be readily incorporated into the real-life video copy detection systems. (C) 2014 Elsevier B.V. All rights reserved.