Shape Perception in Computer Vision

考虑一下为杯子建模的任务，这样计算机视觉系统就可以识别放在相机前的任何杯子。我们如何根据杯子的部件对其进行建模？我们如何从映像中恢复这些部分？我们如何确定一个我们以前从未见过的不同寻常的物体是否可以用作杯子？我们如何利用深度学习的力量来完成这些任务？最后，我们非凡的人类视觉系统如何为这些深度学习解决方案的设计提供信息？在我们最近的工作中，我们展示了局部对称性在人类形状感知中的重要作用，并证明了当输入增加局部对称性信息时，标准深度学习体系的性能可以得到改善，即框架不能单独计算这些信息。我们在两个重要方面扩展了这项工作：1)我们将开发一个可以计算这些有价值的信息的框架；2)我们将把这个模块嵌入到一个更大的端到端的深度学习框架中，用于识别任务，鼓励更具模块的识别管道，更容易解释，并尊重对称性在人类视觉中所起的重要作用。我们的第二个项目解决了三维形状表示学习的问题。具体地说，对于属于某一类别的给定对象的一组3-D图像，我们如何学习该类别对象的表示，该表示可用于生成/识别属于该类别的新示例？在我们之前的工作中，我们学会了将习得的表征梳理开来，或将其“解开”，以区分由于连接(例如，特定老虎腿的不同形状)而在对象中的外部变化和由于类内变化(例如，大型猫科动物家族中的形状变化)而在对象中的内在变化。我们将在表征学习中扩展这一工作，以学习对象的自然部分结构的组成模型，这是一个深深植根于人类和计算机视觉的概念。在我们的第三个项目中，我们不仅仅学习形状的纯粹几何表示法，而是学习更抽象的表示法，这些表示法考虑到对象将如何使用，即它的承受能力。回到我们的杯子例子，壶可以用作杯子，但在任何现代识别系统中都不会被归类为杯子。不是杯子的精确几何形状提供了喝水的任务，而是物体既提供了容器，又提供了倾斜容器倒入液体的手柄。我们扩展了我们之前在学习形状和启示之间的映射方面的工作，以利用基于物理的模拟器和新的新的形状表示的能力，以学习如何抓住对象以促进特定的任务。

Consider the task of modeling a cup, so that a computer vison system can recognize any cup placed in front of a camera. How do we model the cup in terms of its parts? How do we recover those parts from an image? How do we determine if an unusual object we've never seen before can be used as a cup? How do we utilize the power of deep learning to perform these tasks? And finally, how can our remarkable human vision system inform the design of these deep learning solutions? Shedding light on these fundamental questions in shape perception could have a major impact on such downstream tasks as object representation, recognition, and manipulation.  In our recent work, we demonstrated the important role that local symmetry plays in human shape perception, and showed that the performance of a standard deep learning architecture can be improved when the input is augmented with local symmetry information, i.e., the framework cannot compute this information on its own. We extend that work in two important ways: 1) we will develop a framework that can compute this valuable information; and 2) we will embed this module in a larger end-to-end, deep learning framework for recognition tasks, encouraging a more modular recognition pipeline that's more interpretable, and respects the important role that symmetry plays in human vision. Our second project addresses the problem of 3-D shape representation learning. Specifically, for a given a set of 3-D images of objects belonging to a category, how do we learn a representation for that class of objects that can be used to generate/recognize new examples belonging to the class? In our previous work, we learned to tease apart, or "disentangle", the learned representation to differentiate between extrinsic variations in the object due to articulation (e.g., the different configurations of a particular tiger's legs) and intrinsic variations in the object due to within-class variation (e.g., the variations in shape across the family of big cats). We will extend that work in representation learning to learn compositional models of objects in terms of their natural part structure, a concept deeply rooted in both human and conputer vision. In our third project, we move beyond learning purely geometric representations of shape to learn more abstract representations that take into account how the object will be used, i.e., its affordances. Returning to our cup example, a pot could be used as a cup, but would never be categorized as a cup in any modern-day recognition system. It's not the precise geometry of the cup that affords the task of drinking, but the fact that the object affords both containment and a handle with which to tilt the container to pour the liquid. We extend our previous work in learning the mapping between shape and affordance to leverage the power of physics-based simulators and new  shape representations in order to learn how to grasp an object in order to facilitate a particular task.