Looking at People and Web-Scale Image Analysis

点击翻译按钮获取中文摘要

Humans are intensely social, and as such we spend a lot of time looking at people. With a simple glance we perceive pose, actions, interactions, intentions and emotions, all of which are necessary for effective social interaction. Future computers are going to spend as much time looking at people as we do. This will be important for the design of new man-machine interfaces, perceptive consumer devices, surveillance systems, aids for the visually impaired, and myriad systems we have yet to envision.****This proposal aims to develop state-of-the-art models of human shape, appearance, kinematics and dynamics to facilitate video-based analysis of people. This entails new techniques for estimating human pose, motion, and interactions with objects in the environment. The core technologies have myriad possible applications, including those in health care, assessing recovery of locomotion after hip replacement, to assessing the efficiency of movement in high performance sport.****One key to looking at people, and to vision more generally, is learning. Vision relies on sophisticated models of the visual world (e.g., object shape, appearance, motion, etc.), which comprise our prior beliefs that support our ability to make plausible inferences when the image alone only partially constrains the scene (which is true most of the time). And these models are so hard to specify from first principles, that it is often more effective to learn them from data. Doing so entails working with massive image or video corpora, a.k.a. big data.****To facilitate storage and analysis of huge image corpora, we are developing new data structures and algorithms for fast modeling, indexing, search and retrieval of high dimensional data. We recently developed several key technologies that allow us to store billions of images on a single computer, and perform similarity search (finding database items similar to a query image) in a few milliseconds. We plan to continue improving and generalizing such techniques, allowing one to store, retrieve, model and learn (classification and regression) from massive datasets. This will facilitate improved learning in vision, and it will find applications in web-scale search and retrieval applications, like searching all the images on the web based on image content, but using a small fraction of the computational resources that are currently required by companies like Google to perform the same tasks.**