Desktop Action Recognition From First-Person Point-of-View
Desktop Action Recognition From First-Person Point-of-View
复制标题
第一人称视角的桌面动作识别
DOI：
10.1109/tcyb.2018.2806381
复制
发表时间：
2019-05-01
影响因子：
11.8
通讯作者：
Gao, Yue
中科院分区：
文献类型：
--
作者：
Cai, Minjie;Lu, Feng;Gao, Yue
从第一人称视角（自我中心）视频中识别桌面动作是一项重要的任务，因为它在我们的日常生活中无处不在，也是观察手-物交互的理想第一人称视角。然而，以前的研究工作还没有专门针对这项任务的基准。在本文中，我们首先发布了一个使用可穿戴相机记录的日常桌面动作数据集，并将其作为桌面动作识别的基准发布。六名参与者的日常桌面活动用广角头戴式摄像机记录在以自我为中心的视频中。我们特别关注五种涉及手的常见桌面操作。我们提供原始视频数据、帧级的动作注释和像素级的手部蒙版。我们还提出了一种基于手的时空信息来表征不同桌面动作的特征表示。在实验中，我们说明了数据集的统计信息，并评估了不同特征作为基线的动作识别性能。所提出的方法在五个动作类上都取得了令人满意的性能。
Desktop action recognition from first-person view (egocentric) video is an important task due to its omnipresence in our daily life, and the ideal first-person viewing perspective for observing hand-object interactions. However, no previous research efforts have been dedicated on the benchmark of the task. In this paper, we first release a dataset of daily desktop actions recorded with a wearable camera and publish it as a benchmark for desktop action recognition. Regular desktop activities of six participants were recorded in egocentric video with a wide-angle head-mounted camera. In particular, we focus on five common desktop actions in which hands are involved. We provide original video data, action annotations at frame-level, and hand masks at pixel-level. We also propose a feature representation for the characterization of different desktop actions based on the spatial and temporal information of hands. In experiments, we illustrate the statistical information about the dataset, and evaluate the action recognition performance of different features as a baseline. The proposed method achieves promising performance for five action classes.