Understanding Humans in Images and Videos

计算机视觉的一个重大挑战是从视觉数据中理解人。给定一幅图像(或视频)，我们希望建立能够理解其内容并提供关于其中人的丰富描述的计算机系统，例如定位人、识别他们的行为、识别他们的属性(例如性别、职业等)、推断他们与其他人的关系等。针对这一挑战的可靠解决方案将使许多应用成为可能，例如监视、娱乐、网络搜索、人机交互、拟议研究的目标是开发一套知识体系，以促进在视觉数据中理解人，并以具体应用展示技术。这一领域的大部分工作侧重于分类任务。例如，在动作识别中，目标是将图像相应地分类为几组预定义的动作类别。我们的假设是，为了使计算机能够完全理解图像/视频中的人，单一的动作类标签是不够的-我们需要能够提供对视觉数据的更丰富和更深入理解的模型和算法。在这项拟议的研究中，我们将探索一般领域中的三个主题：看人。首先，我们将开发图像中人类动作的分层表示。其次，我们将开发一些方法来将包含一些复杂活动的视频解析成多个粒度的时间段。第三，我们将开发弱监督方法来从视觉数据中发现有意义的中层表示。拟议的研究将提供许多应用技术，例如监控、人机交互、图像/视频检索等。它还将解决计算机视觉和机器学习中的许多基本研究问题。

A grand challenge of computer vision is "understanding people" from visual data. Given an image (or video), we would like to build computer systems that can understand its content and provide rich descriptions about people in it, e.g. localizing people, recognizing their actions, identifying their attributes (e.g. gender, occupation, etc), inferring their relations with other people, etc. A reliable solution to this challenge will enable many applications, e.g. surveillance, entertainment, web search, human computer interaction, etc. The objective of the proposed research is to develop a body of knowledge that facilitates understanding people in visual data and to showcase the technologies with concrete applications.Most of the work in this area focuses on classification tasks. For example, in action recognition, the goal is to classify an image accordingly to several pre-defined sets of action categories. Our hypothesis is that in order to enable computers to fully understand people in images/videos, a single action class label is not enough --- we need models and algorithms that can provide a richer and deeper understanding of the visual data.In this proposed research, we will explore three themes in the general area of "looking at people". First, we will develop hierarchical representation for human actions in images. Second, we will develop methods for parsing a video containing some complex activity into multiple granularity of temporal segments. Third, we will develop weakly-supervised approaches for discovering meaningful mid-level representations from visual data.The proposed research will provide technologies that enable many applications, e.g. surveillance, human computer interactions, image/video retrieval, etc. It will also address many fundamental research questions in computer vision and machine learning.