Understanding Humans in Images and Videos

计算机视觉的一大挑战是从视觉数据中“理解人”。给定一个图像（或视频），我们希望构建能够理解其内容并提供丰富描述的计算机系统，例如定位人，识别他们的行为，识别他们的属性（例如性别，职业等），推断他们与其他人的关系等。一个应对这一挑战的可靠解决方案将使许多应用成为可能，例如监视、娱乐、网络搜索、人机交互等。拟议研究的目标是发展一套知识体系，以方便理解可视化数据中的人，并展示具有具体应用的技术。****这个领域的大部分工作都集中在分类任务上。例如，在动作识别中，目标是根据几个预定义的动作类别集对图像进行分类。我们的假设是，为了使计算机能够完全理解图像/视频中的人，单个动作类标签是不够的——我们需要能够提供更丰富、更深入理解视觉数据的模型和算法。****在这个拟议的研究中，我们将探讨“看人”的一般领域的三个主题。首先，我们将开发图像中人类行为的分层表示。其次，我们将开发将包含一些复杂活动的视频解析为多个时间片段粒度的方法。第三，我们将开发弱监督的方法，从视觉数据中发现有意义的中层表示。****建议的研究将提供能够实现许多应用的技术，例如监控，人机交互，图像/视频检索等。它还将解决计算机视觉和机器学习方面的许多基础研究问题。＊＊＊＊＊＊

A grand challenge of computer vision is "understanding people" from visual data. Given an image (or video), we would like to build computer systems that can understand its content and provide rich descriptions about people in it, e.g. localizing people, recognizing their actions, identifying their attributes (e.g. gender, occupation, etc), inferring their relations with other people, etc. A reliable solution to this challenge will enable many applications, e.g. surveillance, entertainment, web search, human computer interaction, etc. The objective of the proposed research is to develop a body of knowledge that facilitates understanding people in visual data and to showcase the technologies with concrete applications.****Most of the work in this area focuses on classification tasks. For example, in action recognition, the goal is to classify an image accordingly to several pre-defined sets of action categories. Our hypothesis is that in order to enable computers to fully understand people in images/videos, a single action class label is not enough --- we need models and algorithms that can provide a richer and deeper understanding of the visual data.****In this proposed research, we will explore three themes in the general area of "looking at people". First, we will develop hierarchical representation for human actions in images. Second, we will develop methods for parsing a video containing some complex activity into multiple granularity of temporal segments. Third, we will develop weakly-supervised approaches for discovering meaningful mid-level representations from visual data.****The proposed research will provide technologies that enable many applications, e.g. surveillance, human computer interactions, image/video retrieval, etc. It will also address many fundamental research questions in computer vision and machine learning. ******