AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecture
AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecture
复制标题
DOI：
10.1109/iccv.2017.473
复制
发表时间：
2017-04
期刊：
影响因子：
--
通讯作者：
Suman Saha;Gurkirt Singh;Fabio Cuzzolin
中科院分区：
文献类型：
--
作者：
Suman Saha;Gurkirt Singh;Fabio Cuzzolin
动作检测的主要方法只能提供问题的次优解决方案，因为它们依赖于寻求帧级检测，以便稍后在后处理步骤中将它们组合成“动作管”。在本文中，我们从根本上脱离了当前的实践，并朝着设计和实现能够分类和回归整个视频子集的深度网络架构迈出了第一步，从而为动作检测问题提供了真正的最佳解决方案。在这项工作中，特别是，我们提出了一种新的深度网络框架，能够对跨越两个连续视频帧的3D区域建议进行回归和分类，其核心是经典区域建议网络（RPN）的演变。因此，我们的3D-RPN网络能够通过纯粹利用外观来有效地编码动作的时间方面，而不是严重依赖昂贵的流图的方法。所提出的模型是端到端可训练的，并且可以在单个步骤中联合优化动作定位和分类。在测试时，网络预测包含两个连续帧的“微管”，通过一种新算法将其连接成完整的动作管，该算法利用了网络学习的时间编码，并将计算时间缩短了50%。在J-HMDB-21和UCF-101动作检测数据集上的令人鼓舞的结果表明，我们的模型在纯粹依赖外观时确实优于最先进的模型。
Dominant approaches to action detection can only provide sub-optimal solutions to the problem, as they rely on seeking frame-level detections, to later compose them into ‘action tubes’ in a post-processing step. With this paper we radically depart from current practice, and take a first step towards the design and implementation of a deep network architecture able to classify and regress whole video subsets, so providing a truly optimal solution of the action detection problem. In this work, in particular, we propose a novel deep net framework able to regress and classify 3D region proposals spanning two successive video frames, whose core is an evolution of classical region proposal networks (RPNs). As such, our 3D-RPN net is able to effectively encode the temporal aspect of actions by purely exploiting appearance, as opposed to methods which heavily rely on expensive flow maps. The proposed model is end-to-end trainable and can be jointly optimised for action localisation and classification in a single step. At test time the network predicts ‘micro-tubes’ encompassing two successive frames, which are linked up into complete action tubes via a new algorithm which exploits the temporal encoding learned by the network and cuts computation time by 50%. Promising results on the J-HMDB-21 and UCF-101 action detection datasets show that our model does outperform the state-of-the-art when relying purely on appearance.