RI: Small: Functional Object Modeling

人类可以毫不费力地推断场景中对象的功能并与它们进行交互。在面对橱柜门时，人类可以立即识别把手的位置，了解如何与它交互，并预测交互将在场景中引起什么。在充分考虑交互和动态的情况下理解对象功能是计算中必不可少但缺少的能力。对这些对象功能的理解有助于在广泛范围内的新应用。首先，该项目改变了机器人如何影响我们的社会。救援机器人能够在不受控制的，意外的和灾难性的环境中工作，以帮助被困受害者的生命。家用机器人更深入地参与我们的日常活动，帮助我们从家务到照顾老龄化社会的老人。该项目能够轻松构建3D打印的功能对象模型，该技术已经通过在艺术，时尚，人类医疗保健，建筑等领域的应用彻底改变了世界。随着虚拟现实和增强现实的出现，与虚拟环境的交互变得越来越普遍，该项目还可以轻松地为虚拟资产添加“生命”，并创建“活的”虚拟环境，以更好地进行培训和教育。本研究研究了构建，学习和推断功能对象模型的计算框架和算法。以下两个关键观察结果是该项目的核心。首先，对象是为最佳功能而设计的，仔细的基于物理的推理应该会自动揭示信息。例如，如果橱柜门的把手在中间，则门应向前滑动以最小化所需的力和扭矩的量。其次，功能在对象类别中是通用的。例如，打开行为与橱柜门、洗衣机、烤箱和冰箱没有根本区别。该方法通过将分割、运动估计和运动学推理问题集成到一个新的物理驱动的公式中来发现对象功能。研究团队开发算法，从图像、RGBD图像和3D模型中学习和推断对象功能。该项目开辟了一个新的研究前沿，功能对象建模，以及引入了一个原则性的新的通用计算框架，结合物理学。

Humans can effortlessly infer functionalities of objects in a scene and interact with them. In facing a cabinet door, a human can instantly identify the location of a handle, understand how to interact with it, and predict what the interaction would cause in a scene. Understanding object functionality with full consideration of interactions and dynamics is an essential but missing capability in computing. The understanding of such object functionalities facilitates novel applications in a broad spectrum. First, the project changes how robots influences our society. Rescue robots becomes able to operate in uncontrolled, unexpected, and disastrous environments to help the lives of trapped victims. Home-robots becomes more deeply involved in our daily activities, helping us from household chores to the care of the elderly in the aging societies. The project enables to construct functional object models easily for 3D printing, a technology that is already revolutionizing the world through applications in art, fashion, human health-care, construction, and more. At the emergence of virtual reality and augmented reality, where interactions with the virtual environments are becoming more ubiquitous, this project also enables to easily add "lives" to virtual assets and create "live" virtual environments for better training and education.This research studies a computational framework and algorithms to construct, learn, and infer functional object models. The following two key observations are at the heart of the project. First, objects are designed for optimal functionality, and careful physics-based reasoning should automatically reveal the information. For example, if the handle of a cabinet door is in the middle, the door should slide-forward to minimize the amount of necessary forces and torques. Second, functionalities are universal across object categories. For example, an act of opening is not fundamentally different from cabinet doors, laundry machines, ovens, to refrigerators. The approach discovers object functionality by integrating segmentation, motion estimation, and kinematic inference problems into a novel physics-driven formulation. The research team develops algorithms to learn and infer object functionalities from images, RGBD images, and 3D models. The project opens up a new frontier of research, functional object modeling, as well as introduces a principled new yet general computational framework incorporating physics.