Shape Perception in Computer Vision

点击翻译按钮获取中文摘要

Consider the task of modeling a cup, so that a computer vison system can recognize any cup placed in front of a camera. How do we model the cup in terms of its parts? How do we recover those parts from an image? How do we determine if an unusual object we've never seen before can be used as a cup? How do we utilize the power of deep learning to perform these tasks? And finally, how can our remarkable human vision system inform the design of these deep learning solutions? Shedding light on these fundamental questions in shape perception could have a major impact on such downstream tasks as object representation, recognition, and manipulation.  In our recent work, we demonstrated the important role that local symmetry plays in human shape perception, and showed that the performance of a standard deep learning architecture can be improved when the input is augmented with local symmetry information, i.e., the framework cannot compute this information on its own. We extend that work in two important ways: 1) we will develop a framework that can compute this valuable information; and 2) we will embed this module in a larger end-to-end, deep learning framework for recognition tasks, encouraging a more modular recognition pipeline that's more interpretable, and respects the important role that symmetry plays in human vision. Our second project addresses the problem of 3-D shape representation learning. Specifically, for a given a set of 3-D images of objects belonging to a category, how do we learn a representation for that class of objects that can be used to generate/recognize new examples belonging to the class? In our previous work, we learned to tease apart, or "disentangle", the learned representation to differentiate between extrinsic variations in the object due to articulation (e.g., the different configurations of a particular tiger's legs) and intrinsic variations in the object due to within-class variation (e.g., the variations in shape across the family of big cats). We will extend that work in representation learning to learn compositional models of objects in terms of their natural part structure, a concept deeply rooted in both human and conputer vision. In our third project, we move beyond learning purely geometric representations of shape to learn more abstract representations that take into account how the object will be used, i.e., its affordances. Returning to our cup example, a pot could be used as a cup, but would never be categorized as a cup in any modern-day recognition system. It's not the precise geometry of the cup that affords the task of drinking, but the fact that the object affords both containment and a handle with which to tilt the container to pour the liquid. We extend our previous work in learning the mapping between shape and affordance to leverage the power of physics-based simulators and new  shape representations in order to learn how to grasp an object in order to facilitate a particular task.