Generalized Adversarial and Hierarchical Co-occurrence Network based Synthetic Skeleton Generation and Human Identity Recognition
Generalized Adversarial and Hierarchical Co-occurrence Network based Synthetic Skeleton Generation and Human Identity Recognition
复制标题
DOI：
10.1109/ijcnn55064.2022.9892887
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
J. Zalameda;Brady Kruse;Alexander M. Glandon;Megan A. Witherow;Sachin Shetty;K. Iftekharuddin
中科院分区：
文献类型：
--
作者：
J. Zalameda;Brady Kruse;Alexander M. Glandon;Megan A. Witherow;Sachin Shetty;K. Iftekharuddin
人体骨骼数据提供了可用于人类身份和活动识别的相对关节位置的紧凑、低噪声表示。层次化共生网络(HCN)由于能够在网络中考虑卷积运算中关节间的相关性而被用于人类活动识别。HCN具有较好的识别精度，但需要大量的样本进行训练。获取这种大规模数据可能既耗时又昂贵，这促使在HCN中生成用于数据增强的合成骨架数据。提出了一种集成辅助分类器生成对抗网络(AC-GAN)和骨架评估与增强身份识别(AAIRS)的混合框架的新方法。AAIRS方法对合成的三维运动捕捉骨架视频进行生成和评估，然后进行身份识别。由AC-GAN的生成器组件产生的合成骨架数据使用从HCN分类器输出计算的初始分数启发的真实感度量来评估。我们研究了增加训练集中合成样本的百分比对HCN性能的影响。在人工数据增强之前，对于9类人的身份识别，我们在10倍交叉验证中获得了74.49%的HCN性能。在50%-50%的合成-真实混合的情况下，我们获得了78.22%的平均准确率，显著地超过了基线的HCN性能。该框架证明了将合成数据生成体系结构与分层共现特征学习相结合用于身份识别的可行性。
Human skeleton data provides a compact, low noise representation of relative joint locations that may be used in human identity and activity recognition. Hierarchical Co-occurrence Network (HCN) has been used for human activity recognition because of its ability to consider correlation between joints in convolutional operations in the network. HCN shows good identification accuracy but requires a large number of samples to train. Acquisition of this large-scale data can be time consuming and expensive, motivating synthetic skeleton data generation for data augmentation in HCN. We propose a novel method that integrates an Auxiliary Classifier Generative Adversarial Network (AC-GAN) and HCN hybrid framework for Assessment and Augmented Identity Recognition for Skeletons (AAIRS). The proposed AAIRS method performs generation and evaluation of synthetic 3-dimensional motion capture skeleton videos followed by human identity recognition. Synthetic skeleton data produced by the generator component of the AC-GAN is evaluated using an Inception Score-inspired realism metric computed from the HCN classifier outputs. We study the effect of increasing the percentage of synthetic samples in the training set on HCN performance. Before synthetic data augmentation, we achieve 74.49% HCN performance in 10-fold cross validation for 9-class human identification. With a synthetic-real mixture of 50%-50%, we achieve 78.22% mean accuracy, significantly $(\mathrm{p} < 0.05)$ outperforming the baseline HCN performance. The proposed framework demonstrates the feasibility of combining a synthetic data generation architecture with hierarchical co-occurrence feature learning for human identity recognition.