Hierarchical models for the recognition of human activities in video data

用于识别视频数据中人类活动的分层模型

• 批准号: 311269674
• 负责人: Dr. Hildegard Kühne
• 金额: --
• 依托单位: Institut für Informatik
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2016
• 资助国家: 德国
• 起止时间: 2015-12-31 至 2019-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/311269674?language=en
• 关键词: Hierarchical; models; recognition; human; activities

With the growing amount of video data recorded and distributed everyday there is also a growing need for automated processing. To address the complexity of these data, video-based action recognition needs to advance from simple classification of pre-segmented clips with only one clearly defined activity towards the analysis of longer video sequences. First approaches to deal with the recognition of those sequences have been made, but they usually consider stringent timelines without regarding the specific hierarchical nature of human activities. The proposed project fills this gap by focusing on the analysis of temporal hierarchies of human activities in video sequences. It is assumed that human activities are made up of basic building blocks that can be subsumed over several stages to form larger, meaningful activities. In this context, this work aims at the exploration of hierarchical temporal structures for video-based action recognition with the goal is to analyze and recognize complex human activities in videos. To transfer hierarchical models to real action recognition scenarios, a three-stage approach is proposed. First, a bottom-up recognition system for human actions based on small temporal entities will be built. The entities will be concatenated and pooled over several temporal layers to build a high-level representation. The system will be built on generative models, as they have been successfully applied in the context of similar problems. Second, to avoid the task of labeling data, semi- and unsupervised training procedures will be implemented and evaluated. Therefore, existing unlabeled training material will be segmented and clustered, either in a semi- or unsupervised way and the resulting units will form the input for an automatically generated grammar and language format. The resulting training procedure should be able to segment a given training set into small parts, to combine them by clustering, and to build an overall representation of the activity domain by the generation of a grammar based on the defined entities. Third, as the system provides for generative, temporal modeling over time, those properties will be exploited with regard to its potential in integrating context knowledge: the generative nature of the overall model allows easy integration of context in the form of probability distributions at any stage of the recognition process and the temporal modeling provides not only an integration of context but also the assessment of context, e.g. in the form of object states, over time. The overall final system should provide both hierarchical recognition and analysis of human actions with regard to environmental context as well as the training routines needed to apply this model to a large variety of different datasets and application domains. We hope that the system will provide new ways to deal with the challenges of analyzing complex activities over time and that it will allow new applications in this field.

随着每天记录和分发的视频数据量不断增加，对自动化处理的需求也越来越大。为了解决这些数据的复杂性，基于视频的动作识别需要从仅具有一个明确定义的活动的预分割片段的简单分类推进到更长视频序列的分析。处理这些序列的识别的第一种方法已经提出，但它们通常考虑严格的时间表，而不考虑人类活动的具体层次性质。拟议的项目填补了这一空白，专注于分析的时间层次的人类活动的视频序列。它假设人类活动是由基本的积木组成的，这些积木可以被归入几个阶段，形成更大的、有意义的活动。在这种情况下，这项工作的目的是探索基于视频的动作识别的层次时间结构，其目标是分析和识别视频中复杂的人类活动。为了将层次模型转换为真实的动作识别场景，提出了一种三阶段方法。首先，将建立一个基于小时间实体的自底向上的人类动作识别系统。这些实体将在几个时间层上连接和合并，以构建高级表示。该系统将建立在生成模型，因为它们已成功地应用于类似的问题。其次，为了避免标记数据的任务，将实施和评估半监督和无监督的训练程序。因此，现有的未标记的训练材料将以半监督或无监督的方式被分割和聚类，并且所得到的单元将形成用于自动生成的语法和语言格式的输入。所得到的训练过程应该能够将给定的训练集分割成小部分，通过聚类将它们联合收割机组合，并且通过基于所定义的实体生成语法来构建活动域的总体表示。第三，由于系统提供随时间推移的生成性时间建模，因此将利用这些属性来整合上下文知识：整个模型的生成特性允许在识别过程的任何阶段以概率分布的形式容易地集成上下文，并且时间建模不仅提供上下文的集成而且提供上下文的评估，例如以对象状态的形式随时间变化。整个最终系统应该提供人类行为的分层识别和分析，以及将该模型应用于各种不同的数据集和应用领域所需的训练例程。我们希望该系统将提供新的方法来应对随着时间的推移分析复杂活动的挑战，并将允许在这一领域的新应用。