CAREER: Discriminative Spatiotemporal Models for Recognizing Humans, Objects, and their Interactions

计算机视觉的目标之一是建立一个能够看到人并识别他们活动的系统。人类的行为很少是孤立地进行的——周围的环境、附近的物体和附近的人都会影响所执行活动的性质。例如“吃东西”和“握手”等动作。这个项目的研究目标是接近人类在理解由人-对象和人-人互动定义的活动视频中的表现。这个项目利用结构化的上下文表示来预测给定的时空数据。它通过将最近成功的对象识别工作扩展到时空域，引入时空分组和上下文建模的扩展来实现这一点。视频能够提取静态图像中缺少的额外动态线索，但这带来了额外的计算负担，可以通过近似解析和大规模判别学习的算法创新来解决。为了将活动识别建立在坚实的定量基础上，所提出的模型使用基于日常生活活动（ADL）和医学和人类学社区的人类邻近模型的具体指标进行评估。例子包括自动监测中风患者与日常物品互动的系统，以及在紧急演习期间自动分析危机反应小组互动的系统。这个项目产生了非脚本的、真实的、标记的动作识别数据集，对整个研究界都有好处。

One of the goals of computer vision is to build a system that can see people and recognize their activities. Human actions are rarely performed in isolation -- the surrounding environment, nearby objects, and nearby humans affect the nature of the performed activity.Examples include actions such as "eating" and "shaking hands." The research goal of this project is to approach human performance in understanding videos of activities defined by human-object and human-human interactions.This project makes use of structured, contextual representations to make predictions given spatiotemporal data. It does so by extending recent successful work on object recognition to the space-time domain, introducing extensions for spatiotemporal grouping and contextual modeling. Video enables the extraction of additional dynamic cues absent in static images, but this poses additional computational burdens that are addressed through algorithmic innovations for approximate parsing and large-scale discriminative learning.To place activity recognition on firm quantitative ground, the proposed models are evaluated using concrete metrics based on activities of daily living (ADL) and human proxemic models from the medical and anthropological communities. Examples include systems for automated monitoring of stroke patients interacting with everyday objects and automated analysis of crisis response team interactions during emergency drills. This project produces non-scripted, real-world, labeled action recognition datasets, of benefit to the research community as a whole.