Shifting Perspective to See Difference: A Novel Multi-view Method for Skeleton based Action Recognition
Shifting Perspective to See Difference: A Novel Multi-view Method for Skeleton based Action Recognition
复制标题
DOI：
10.1145/3503161.3548210
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Ruijie Hou;Yanran Li;Ningyu Zhang-;Yulin Zhou;Xiaosong Yang;Zhao Wang
中科院分区：
文献类型：
--
作者：
Ruijie Hou;Yanran Li;Ningyu Zhang-;Yulin Zhou;Xiaosong Yang;Zhao Wang
由于其复杂的动态特性，基于以太网的人类动作识别是一个长期的挑战。动力学的一些细粒度细节在分类中起着至关重要的作用。现有的工作主要集中在设计具有更复杂的邻接矩阵的增量神经网络来捕捉关节关系的细节。然而，他们仍然很难区分具有大致相似的运动模式但属于不同类别的动作。有趣的是，我们发现，运动模式的细微差异可以显着放大，并变得容易为观众区分通过指定的视图方向，在此属性还没有得到充分的探索之前。与以前的工作不同，我们提出了一个概念上简单而有效的多视图策略，从动态视图功能的集合中识别动作，从而提高了性能。具体来说，我们设计了一个新的锚定建议（SAP）模块，其中包含一个多头结构，以学习一组视图。对于不同视图的特征学习，我们引入了一种新的角度表示来转换不同视图下的动作，并将转换馈送到基线模型中。我们的模块可以与现有的动作分类模型无缝地工作。结合基线模型，我们的SAP模块在许多具有挑战性的基准测试中表现出明显的性能提升。此外，全面的实验表明，我们的模型始终击败了最先进的，仍然是有效的和强大的，特别是在处理损坏的数据。相关代码将在https://github.com/ideal-idea/SAP上提供
Skeleton-based human action recognition is a longstanding challenge due to its complex dynamics. Some fine-grain details of the dynamics play a vital role in classification. The existing work largely focuses on designing incremental neural networks with more complicated adjacent matrices to capture the details of joints relationships. However, they still have difficulties distinguishing actions that have broadly similar motion patterns but belong to different categories. Interestingly, we found that the subtle differences in motion patterns can be significantly amplified and become easy for audience to distinct through specified view directions, where this property haven't been fully explored before. Drastically different from previous work, we boost the performance by proposing a conceptually simple yet effective Multi-view strategy that recognizes actions from a collection of dynamic view features. Specifically, we design a novel Skeleton-Anchor Proposal (SAP) module which contains a Multi-head structure to learn a set of views. For feature learning of different views, we introduce a novel Angle Representation to transform the actions under different views and feed the transformations into the baseline model. Our module can work seamlessly with the existing action classification model. Incorporated with baseline models, our SAP module exhibits clear performance gains on many challenging benchmarks. Moreover, comprehensive experiments show that our model consistently beats down the state-of-the-art and remains effective and robust especially when dealing with corrupted data. Related code will be available on https://github.com/ideal-idea/SAP