CAREER: Physical Object Modeling for Intelligent Systems

人工智能(AI)和计算机视觉的最新进展表明，建造能够看到并解释图像或视频中的内容的机器是可能的。然而，为了让机器像人类一样与世界进行物理互动，它们需要了解的不仅仅是“什么是哪里”，还需要了解世界包含具有不同属性的物理对象，如它们的材料、形状、密度和多感官特征(它们如何听和感觉)。在当前的人工智能系统中，物理对象建模问题在很大程度上被忽视了。这个项目旨在介绍一些系统和方法，这些系统和方法可以让我们捕捉和模拟各种不同的、多感官的物理物体，以帮助智能机器更好地理解世界并与世界互动。研究人员将把这项研究的成果整合到课程开发、学生建议和社区基础设施开发中；他们还将与教育和非营利组织合作，向未被充分代表的学生教授人工智能、视觉、图形学和机器人学。在这个项目中，研究人员将专注于多感官物理对象的建模和广泛应用。技术计划的核心是基于物理的对象表示，它编码了世界上基于物理的、以对象为中心的先验，包括对象纹理、材料、几何、关节、物理属性和多感官特征的概念。围绕这些表示，研究团队将开发(1)利用集成硬件和学习捕获物理对象的数据集的新系统；(2)利用这些对象执行计算机视觉、图形和机器人任务的新方法，例如多感官感知和重建、对象和场景生成和编辑、动力学建模和模拟以及机器人交互和操纵；(3)研究物理对象感知和交互的神经认知基础的新范式，具有为智能机器提供更多类似人类能力的长期潜力。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advancements in artificial intelligence (AI) and computer vision have shown the possibility of building machines that can see and explain what is in an image or video. However, for machines to physically interact with the world just like humans, they need to understand more than “what is where” but also that the world contains physical objects with distinct properties, such as their material, shape, density, and multi-sensory characteristics (how they sound and feel). This problem of physical object modeling remains largely ignored in current AI systems. This project aims to introduce systems and methods that may allow us to capture and model diverse, multi-sensory, physical objects to help intelligent machines better understand and interact with the world. Researchers will integrate findings from this research into course development, student advising, and community infrastructure development; they will also partner with educational and non-profit organizations to teach AI, vision, graphics, and robotics to underrepresented students.In this project, investigators will focus on the modeling and extensive applications of multi-sensory physical objects. At the core of the technical plan is a physics-based object representation, which encodes physics-based, object-centric priors in the world, including the notion of object texture, material, geometry, articulation, physical properties, and multi-sensory characteristics. Centered around such representations, the team of researchers will develop (1) new systems that exploit integrated hardware and learning to capture datasets of physical objects; (2) new methods that leverage these objects for computer vision, graphics, and robotics tasks, such as multi-sensory perception and reconstruction, object and scene generation and editing, dynamics modeling and simulation, and robotic interaction and manipulation; (3) new paradigms that study the neurocognitive bases of physical object perception and interaction, with the long-term potential of providing intelligent machines with more human-like capabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.