CAREER: Discriminative Spatiotemporal Models for Recognizing Humans, Objects, and their Interactions

计算机视觉的目标之一是建立一个能够看到人并识别他们活动的系统。人类的行为很少是孤立地进行的--周围的环境、附近的物体和附近的人类会影响所执行的活动的性质。例如，“吃东西”和“握手”之类的动作。这个项目的研究目标是探讨人类在理解由人-对象和人-人交互定义的活动视频方面的表现。该项目利用结构化的上下文表征来预测给定的时空数据。它通过将最近在物体识别方面的成功工作扩展到时空领域，引入了时空分组和上下文建模的扩展来做到这一点。视频能够提取静态图像中不存在的额外动态线索，但这带来了额外的计算负担，这些计算负担通过近似解析和大规模区分学习的算法创新来解决。为了将活动识别建立在坚实的量化基础上，我们使用基于日常生活活动(ADL)和来自医学和人类学社区的人类前驱模型的具体度量来评估所提出的模型。例如，用于自动监测中风患者与日常物品互动的系统，以及在紧急演习期间自动分析危机应对小组互动的系统。这个项目产生了非脚本的、真实世界的、标记的动作识别数据集，对整个研究界都有好处。

One of the goals of computer vision is to build a system that can see people and recognize their activities. Human actions are rarely performed in isolation -- the surrounding environment, nearby objects, and nearby humans affect the nature of the performed activity.Examples include actions such as "eating" and "shaking hands." The research goal of this project is to approach human performance in understanding videos of activities defined by human-object and human-human interactions.This project makes use of structured, contextual representations to make predictions given spatiotemporal data. It does so by extending recent successful work on object recognition to the space-time domain, introducing extensions for spatiotemporal grouping and contextual modeling. Video enables the extraction of additional dynamic cues absent in static images, but this poses additional computational burdens that are addressed through algorithmic innovations for approximate parsing and large-scale discriminative learning.To place activity recognition on firm quantitative ground, the proposed models are evaluated using concrete metrics based on activities of daily living (ADL) and human proxemic models from the medical and anthropological communities. Examples include systems for automated monitoring of stroke patients interacting with everyday objects and automated analysis of crisis response team interactions during emergency drills. This project produces non-scripted, real-world, labeled action recognition datasets, of benefit to the research community as a whole.