Learning discriminative space-time actions from weakly labelled videos
Learning discriminative space-time actions from weakly labelled videos
复制标题
DOI：
10.5244/c.26.123
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
Michael Sapienza;Fabio Cuzzolin;Philip H. S. Torr
中科院分区：
文献类型：
--
作者：
Michael Sapienza;Fabio Cuzzolin;Philip H. S. Torr
当前最先进的动作分类方法从动作展开的整个视频剪辑中提取特征表示，然而，该表示可能包括在多个动作类之间共享的不相关的场景上下文和动作。例如，可以在行走时执行挥手动作，然而，如果行走动作和场景上下文出现在其他动作类中，则它们不应该被包括在挥动动作分类器中。在这项工作中，我们提出了一个动作分类框架，在这个框架中，由于人工标记海量视频数据集的困难，在弱监督环境下学习更多的区分动作子集。学习的模型用于同时对视频剪辑进行分类并将动作定位到给定的时空子体积。在多实例学习框架中，每个子卷被转换为一个袋子特征(BoF)实例，而该实例又被用于学习其类成员资格。我们定量地证明，即使在单个固定大小的子集上，我们提出的算法的分类性能在大多数性能指标上都优于最先进的BoF基线，并且显示出在最具挑战性的视频数据集上进行时空动作定位的前景。
Current state-of-the-art action classification methods extract feature representations from the entire video clip in which the action unfolds, however this representation may include irrelevant scene context and movements which are shared amongst multiple action classes. For example, a waving action may be performed whilst walking, however if the walking movement and scene context appear in other action classes, then they should not be included in a waving movement classifier. In this work, we propose an action classification framework in which more discriminative action subvolumes are learned in a weakly supervised setting, owing to the difficulty of manually labelling massive video datasets. The learned models are used to simultaneously classify video clips and to localise actions to a given space-time subvolume. Each subvolume is cast as a bag-offeatures (BoF) instance in a multiple-instance-learning framework, which in turn is used to learn its class membership. We demonstrate quantitatively that even with single fixedsized subvolumes, the classification performance of our proposed algorithm is superior to the state-of-the-art BoF baseline on the majority of performance measures, and shows promise for space-time action localisation on the most challenging video datasets.