EAGER: Modeling and Recognizing Collective Activities

本项目探索了一个学习集体活动通用模型的新原则框架。集体活动的例子有：人们交谈；一群斑马正在躲避狮子。这些模型依次用于检测、分类和分割活动，以及识别与视频序列中的集体行为不同的活动。本项目开展的研究与以往对行动分类的研究明显不同，以往的研究是孤立地考虑个体来分析活动。此外，与目前的许多贡献不同，它的目标是在无限制的条件下工作，如动态杂乱的背景，移动，单目和未校准的相机。本项目的主要智力贡献是：i)基于随机森林的学习方案，能够自适应地表征个体的连贯行为，从而实现集体活动的判别分类。这种学习方案也适用于使用上下文的其他视觉识别任务（例如，场景和物体识别）；ii)一种基于关系依赖网络的方法，用于分割不同的集体活动并发现异常活动。该项目可以为解决高级视觉问题提供关键的构建模块，例如建模人/动物与对象之间的交互，构建人/动物活动的本体，建模复杂的人/动物行为。这项研究有可能在机器人和导航等战略领域发挥变革性作用。它还为分析和研究生物学（昆虫、动物）或生物医学（细胞）中典型的时空集体行为提供了重要工具。

This project explores a novel principled framework for learning generic models of collective activities. Examples of collective activities are: people talking; a group of zebras escaping from a lion. Such models are used, in turn, for detecting, classifying, and segmenting activities as well as indentifying activities that differ from the collective behavior from videos sequences. Research developed in this project is distinctly different from previous research on action classification wherein activities are analyzed by considering individuals in isolation. Furthermore, unlike many current contributions, the aim is to work under unrestrictive conditions such as dynamic cluttered background, moving, monocular and un-calibrated cameras.Key intellectual contributions of this project are: i) a learning scheme based on Random Forest that is able to adaptively characterize the coherent behavior of individuals, thus enabling discriminative classification of collective activities. This learning scheme is also relevant to other visual recognition tasks using context (e.g., scene and object recognition); ii) a methodology based on Relational Dependency Networks for segmenting different collective activities and discovering anomalous ones. This project can provides critical building blocks toward addressing high level visual problems such as modeling the interaction between humans/animals and objects, constructing an ontology of human/animal activities, modeling complex human/animal behaviors. This research has a potential to play a transformative role in strategic areas such as robotics and navigation. It also provides a crucial tool for analyzing and studying typical spatial-temporal collective behaviors in biology (insects, animals) or biomedicine (cells).