Understanding Humans in Images and Videos

点击翻译按钮获取中文摘要

A grand challenge of computer vision is "understanding people" from visual data. Given an image (or video), we would like to build computer systems that can understand its content and provide rich descriptions about people in it, e.g. localizing people, recognizing their actions, identifying their attributes (e.g. gender, occupation, etc), inferring their relations with other people, etc. A reliable solution to this challenge will enable many applications, e.g. surveillance, entertainment, web search, human computer interaction, etc. The objective of the proposed research is to develop a body of knowledge that facilitates understanding people in visual data and to showcase the technologies with concrete applications. Most of the work in this area focuses on classification tasks. For example, in action recognition, the goal is to classify an image accordingly to several pre-defined sets of action categories. Our hypothesis is that in order to enable computers to fully understand people in images/videos, a single action class label is not enough --- we need models and algorithms that can provide a richer and deeper understanding of the visual data. In this proposed research, we will explore three themes in the general area of "looking at people". First, we will develop hierarchical representation for human actions in images. Second, we will develop methods for parsing a video containing some complex activity into multiple granularity of temporal segments. Third, we will develop weakly-supervised approaches for discovering meaningful mid-level representations from visual data. The proposed research will provide technologies that enable many applications, e.g. surveillance, human computer interactions, image/video retrieval, etc. It will also address many fundamental research questions in computer vision and machine learning.