CAREER: Digitize and Simulate the Large Physical World via Knowledge-Grounded Scene Representation

职业：通过基于知识的场景表示对大型物理世界进行数字化和模拟

• 批准号: 2340254
• 负责人: Shenlong Wang
• 金额: $ 60万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-03-01 至 2029-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2340254&HistoricalAwards=false
• 关键词: CAREER; Digitize; Simulate; Large; Physical

Numerous fields, such as agricultural robotics, necessitate that humans and intelligent agents have a shared understanding of the physical world to be able to make informed decisions. This requires that intelligent machines, to effectively assist humans, understand the physical world, simulate various possibilities, and predict and evaluate outcomes. Current computer vision systems, which largely focus on understanding and modeling observed phenomena, fall short of these capabilities. Similarly, domain-specific simulators fail to achieve this goal as they lack real-world grounding. This project aims to construct AI systems capable of creating a digital replica of the large 3D world and faithfully simulating various counterfactual scenarios, thereby enabling users to assess the outcomes of different decisions and actions. Central to this work is an actionable, knowledge-grounded scene representation, facilitating real-world modeling and simulation. The project has the potential to amplify advancements in various disciplines, such as robotics and agriculture. This project aims to achieve this goal via four directions that advance digitizing and simulating the physical world. To achieve this goal, the project offers a framework that will (i) create an actionable scene representation from real-world videos; (ii) infer physical parameters of the digital world from visual observations; (iii) perform physics-grounded simulation over the digital twin that enables the creation of realistic and accurate counterfactuals; and (iv) integrate the digital twin with domain knowledge and modeling for various applications. Based on the research studies, the project will develop a digital twin education platform that can be applied in undergraduate and graduate education, mentoring, and K-12 outreach activities to engage the next generation of researchers in computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

许多领域，如农业机器人，需要人类和智能代理对物理世界有共同的理解，以便能够做出明智的决策。这就要求智能机器能够有效地帮助人类，理解物理世界，模拟各种可能性，并预测和评估结果。目前的计算机视觉系统主要集中在理解和建模观察到的现象，缺乏这些能力。同样，特定领域的模拟器也无法实现这一目标，因为它们缺乏现实世界的基础。该项目旨在构建能够创建大型3D世界的数字副本并忠实地模拟各种反事实场景的人工智能系统，从而使用户能够评估不同决策和行动的结果。这项工作的核心是一个可操作的，以知识为基础的场景表示，促进现实世界的建模和仿真。该项目有可能扩大各种学科的进步，如机器人和农业。该项目旨在通过推进数字化和模拟物理世界的四个方向来实现这一目标。为了实现这一目标，该项目提供了一个框架，该框架将（i）从真实世界的视频中创建可操作的场景表示;（ii）从视觉观察中推断数字世界的物理参数;（iii）在数字孪生模型上执行基于物理的模拟，从而能够创建逼真和准确的反事实;以及（iv）将数字孪生模型与领域知识和各种应用建模相结合。基于研究成果，该项目将开发一个数字孪生教育平台，可应用于本科生和研究生教育、指导和K-12外展活动，以吸引下一代研究人员参与计算。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。