CAREER: Physical Object Modeling for Intelligent Systems

人工智能（AI）和计算机视觉的最新进展表明，有可能建造能够看到和解释图像或视频中内容的机器。然而，对于像人类一样与世界进行物理交互的机器来说，它们需要了解的不仅仅是“什么在哪里”，而且还需要了解世界包含具有不同属性的物理对象，例如它们的材料、形状、密度和多感官特征（它们的声音和感觉如何）。物理对象建模的这个问题在当前的人工智能系统中仍然被忽视。该项目旨在引入系统和方法，使我们能够捕获和建模多样化的，多感官的物理对象，以帮助智能机器更好地理解和与世界互动。研究人员将把这项研究的结果整合到课程开发、学生咨询和社区基础设施建设中;他们还将与教育和非营利组织合作，向代表性不足的学生教授人工智能、视觉、图形和机器人技术。在这个项目中，研究人员将专注于多感官物理对象的建模和广泛应用。技术计划的核心是基于物理的对象表示，它编码了世界上基于物理的、以对象为中心的先验，包括对象纹理、材料、几何形状、接合、物理属性和多感官特征的概念。围绕这些表示，研究人员团队将开发（1）利用集成硬件和学习来捕获物理对象数据集的新系统;（2）利用这些对象进行计算机视觉，图形和机器人任务的新方法，例如多感官感知和重建，对象和场景生成和编辑，动力学建模和仿真，以及机器人交互和操纵;（3）研究物理对象感知和交互的神经认知基础的新范式，具有为智能机器提供更像人类的能力的长期潜力。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advancements in artificial intelligence (AI) and computer vision have shown the possibility of building machines that can see and explain what is in an image or video. However, for machines to physically interact with the world just like humans, they need to understand more than “what is where” but also that the world contains physical objects with distinct properties, such as their material, shape, density, and multi-sensory characteristics (how they sound and feel). This problem of physical object modeling remains largely ignored in current AI systems. This project aims to introduce systems and methods that may allow us to capture and model diverse, multi-sensory, physical objects to help intelligent machines better understand and interact with the world. Researchers will integrate findings from this research into course development, student advising, and community infrastructure development; they will also partner with educational and non-profit organizations to teach AI, vision, graphics, and robotics to underrepresented students.In this project, investigators will focus on the modeling and extensive applications of multi-sensory physical objects. At the core of the technical plan is a physics-based object representation, which encodes physics-based, object-centric priors in the world, including the notion of object texture, material, geometry, articulation, physical properties, and multi-sensory characteristics. Centered around such representations, the team of researchers will develop (1) new systems that exploit integrated hardware and learning to capture datasets of physical objects; (2) new methods that leverage these objects for computer vision, graphics, and robotics tasks, such as multi-sensory perception and reconstruction, object and scene generation and editing, dynamics modeling and simulation, and robotic interaction and manipulation; (3) new paradigms that study the neurocognitive bases of physical object perception and interaction, with the long-term potential of providing intelligent machines with more human-like capabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.