Region-based Activity Recognition Using Conditional GAN.
Region-based Activity Recognition Using Conditional GAN.
复制标题
DOI：
10.1145/3123266.3123365
复制
发表时间：
2017-10
期刊：
影响因子：
--
通讯作者：
Burd RS
中科院分区：
文献类型：
--
作者：
Li X;Zhang Y;Zhang J;Chen Y;Li H;Marsic I;Burd RS
我们提出了一种活动识别方法，该方法首先估计活动执行者的位置，并将其与输入数据一起用于活动识别。现有的方法直接取视频帧或整段视频进行特征提取和识别，将分类器视为黑盒。我们的方法首先通过使用条件生成对抗网络（cGAN）生成活动掩码来定位每个输入视频帧中的活动。将生成的掩模附加到输入图像的颜色通道上，并将其输入VGG-LSTM网络进行活动识别。为了测试我们的系统，我们用手动创建的口罩制作了两个数据集，一个包含奥林匹克体育活动，另一个包含创伤复苏活动。我们的系统对每个视频帧进行活动预测，并实现与最先进的系统相当的性能，同时勾勒出活动的位置。我们展示了生成的遮罩如何促进对代表活动的特征的学习，而不是偶然的周围信息。
We present a method for activity recognition that first estimates the activity performer’s location and uses it with input data for activity recognition. Existing approaches directly take video frames or entire video for feature extraction and recognition, and treat the classifier as a black box. Our method first locates the activities in each input video frame by generating an activity mask using a conditional generative adversarial network (cGAN). The generated mask is appended to color channels of input images and fed into a VGG-LSTM network for activity recognition. To test our system, we produced two datasets with manually created masks, one containing Olympic sports activities and the other containing trauma resuscitation activities. Our system makes activity prediction for each video frame and achieves performance comparable to the state-of-the-art systems while simultaneously outlining the location of the activity. We show how the generated masks facilitate the learning of features that are representative of the activity rather than accidental surrounding information.