Image abstraction

点击翻译按钮获取中文摘要

One of the most challenging problems in computer vision is object categorization -- the problem of recognizing previously unseen objects belonging to known categories. While early object modeling paradigms were 3-D and captured the prototypical shape of an object, the representational gap between such high-level shape models and the low-level image features that could be reliably extracted was wide, and such systems were restricted to highly constrained scenes. Over the past 20 years, the recognition community moved away from modeling the 3-D prototypical shape of an object to modeling the specific 2-D appearance of the object, supporting the recognition of highly complex, highly textured objects. Unfortunately, appearance-based features such as texture and colour are seldom generic, and while different members of a category may have similar prototypical shape, they may have dissimilar appearance. As a result, an appearance-based categorical object model does a poor job of modeling previously unseen members of the same category that have significantly different appearance. Shape is now making a comeback in the object categorization community, but is typically used to model the specific, as opposed to prototypical, shape of an object. The wide representational gap still remains, and can only be bridged through the process of image abstraction: perceptually grouping together image features that belong to the same object or object part, and simplifying each such group using an appropriate low-dimensional, or reduced, model. This research program addresses the problem of image abstraction on multiple fronts, including the extraction and grouping of symmetric parts from a 2-D scene, the recovery of 3-D surfaces and volumetric parts from a 2-D scene, and learning the semantics of objects and actions in images and videos through language-vision integration. I believe that the problem of image abstraction is the most challenging and most important problem facing the object categorization community. Only when we've defined an appropriate set of 2-D and 3-D abstractions and can recover them from complex, cluttered scenes, can we accomplish true object categorization.