Automatic segmentation and recognition of human activities from observation based on semantic reasoning
Automatic segmentation and recognition of human activities from observation based on semantic reasoning
复制标题
DOI：
10.1109/iros.2014.6943279
复制
发表时间：
2014-11
期刊：
影响因子：
--
通讯作者：
Karinne Ramirez-Amaro;M. Beetz;G. Cheng
中科院分区：
文献类型：
--
作者：
Karinne Ramirez-Amaro;M. Beetz;G. Cheng
从观察中自动分割和识别人类活动通常需要非常复杂和复杂的感知算法。由于这些视觉系统通常需要预处理步骤，因此此类系统不太可能在线实施到物理系统（例如机器人）中。在这项工作中，我们提出并证明，与适当的语义表示的活动，而没有这样复杂的感知系统，它是足够的推断人类活动的视频。首先，我们将提出一种方法来提取基于三个简单的手部动作，即移动，不移动和工具使用的语义规则。此外，使用对象属性ObjectActedOn或ObjectInHand的信息。这些属性封装了当前上下文的信息。上述数据用于训练决策树以获得推理引擎所采用的语义规则。这意味着，我们从视频中提取较低级别的信息，并对预期的人类行为（高级）进行推理。抽象表示的优点是，它允许从人类行为中获得更通用的模型，即使信息是从不同的场景中获得的。结果表明，我们的系统正确分割和识别人类行为的准确率为85%。我们的系统的另一个重要方面是它的可扩展性和对新活动的适应性，这可以按需学习。我们的系统已经完全实现了一个人形机器人，iCub实验验证的性能和鲁棒性，我们的系统在机器人在线执行。
Automatically segmenting and recognizing human activities from observations typically requires a very complex and sophisticated perception algorithm. Such systems would be unlikely implemented on-line into a physical system, such as a robot, due to the pre-processing step(s) that those vision systems usually demand. In this work, we present and demonstrate that with an appropriate semantic representation of the activity, and without such complex perception systems, it is sufficient to infer human activities from videos. First, we will present a method to extract the semantic rules based on three simple hand motions, i.e. move, not move and tool use. Additionally, the information of the object properties either ObjectActedOn or ObjectInHand are used. Such properties encapsulate the information of the current context. The above data is used to train a decision tree to obtain the semantic rules employed by a reasoning engine. This means, we extract lower-level information from videos and we reason about the intended human behaviors (high-level). The advantage of the abstract representation is that it allows to obtain more generic models out of human behaviors, even when the information is obtained from different scenarios. The results show that our system correctly segments and recognizes human behaviors with an accuracy of 85%. Another important aspect of our system is its scalability and adaptability toward new activities, which can be learned on-demand. Our system has been fully implemented on a humanoid robot, the iCub to experimentally validate the performance and the robustness of our system during on-line execution of the robot.