Image abstraction

计算机视觉中最具挑战性的问题之一是对象分类——识别属于已知类别的以前未见过的对象的问题。虽然早期的对象建模范式是3-D的，并且捕获对象的原型形状，但这种高级形状模型与可以可靠提取的低级图像特征之间的表征差距很大，而且这种系统仅限于高度受限的场景。在过去的20年里，识别界从对物体的3-D原型形状建模转向对物体的特定2-D外观建模，以支持对高度复杂、高度纹理物体的识别。不幸的是，基于外观的特征，如纹理和颜色，很少是通用的，虽然一个类别的不同成员可能具有相似的原型形状，但它们可能具有不同的外观。因此，基于外观的分类对象模型在为具有显著不同外观的同一类别中以前未见过的成员建模方面做得很差。形状现在在对象分类社区中卷土重来，但通常用于对对象的特定形状（而不是原型形状）进行建模。广泛的代表性差距仍然存在，只能通过图像抽象过程来弥合：将属于同一对象或对象部分的图像特征感知地分组在一起，并使用适当的低维或简化模型简化每个这样的组。本研究项目解决了多个方面的图像抽象问题，包括从二维场景中提取和分组对称部分，从二维场景中恢复三维表面和体积部分，以及通过语言视觉集成学习图像和视频中对象和动作的语义。我认为图像抽象问题是对象分类界面临的最具挑战性和最重要的问题。只有当我们定义了一组适当的二维和三维抽象，并能从复杂、混乱的场景中恢复它们时，我们才能完成真正的对象分类。

One of the most challenging problems in computer vision is object categorization -- the problem of recognizing previously unseen objects belonging to known categories. While early object modeling paradigms were 3-D and captured the prototypical shape of an object, the representational gap between such high-level shape models and the low-level image features that could be reliably extracted was wide, and such systems were restricted to highly constrained scenes. Over the past 20 years, the recognition community moved away from modeling the 3-D prototypical shape of an object to modeling the specific 2-D appearance of the object, supporting the recognition of highly complex, highly textured objects. Unfortunately, appearance-based features such as texture and colour are seldom generic, and while different members of a category may have similar prototypical shape, they may have dissimilar appearance. As a result, an appearance-based categorical object model does a poor job of modeling previously unseen members of the same category that have significantly different appearance. Shape is now making a comeback in the object categorization community, but is typically used to model the specific, as opposed to prototypical, shape of an object. The wide representational gap still remains, and can only be bridged through the process of image abstraction: perceptually grouping together image features that belong to the same object or object part, and simplifying each such group using an appropriate low-dimensional, or reduced, model. This research program addresses the problem of image abstraction on multiple fronts, including the extraction and grouping of symmetric parts from a 2-D scene, the recovery of 3-D surfaces and volumetric parts from a 2-D scene, and learning the semantics of objects and actions in images and videos through language-vision integration. I believe that the problem of image abstraction is the most challenging and most important problem facing the object categorization community. Only when we've defined an appropriate set of 2-D and 3-D abstractions and can recover them from complex, cluttered scenes, can we accomplish true object categorization.