CAREER: Deep Learning Empowered Nonlinear Deformable Model

职业：深度学习赋能非线性变形模型

• 批准号: 2301040
• 负责人: Yin Yang
• 金额: $ 55万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2301040&HistoricalAwards=false
• 关键词: CAREER; Deep; Learning; Empowered; Nonlinear

Everything in the world deforms, so modeling high-quality deformations becomes a core algorithmic ingredient for serious and realism-driven visual applications such as high-fidelity animation, virtual reality, medical data analysis, surgical simulation, and digital fabrication/prototyping, to name just a few. While deformation has been studied for decades, deformable simulation is notorious for its costly computation. With the rapid development of sophisticated sensing devices and acquisition techniques, the complexities, scales and dimensionalities of the data have grown exponentially, and large-scale geometries are becoming ubiquitous in modern 3D data processing. Even with state-of-the-art hardware, a massive deformable simulation can still take hours, days, or even weeks. In this era of data explosion, increasing demands on both computing efficiency and simulation realism impose unprecedented challenges on this classic computing problem, so game-changing algorithmic techniques for large-scale, complex, and nonlinear deformable models are needed to empower future graphics applications. If successful, this project will not only expand the frontier of physics-based simulation technologies, but also profoundly inspire broader computing communities beyond graphics and enable a variety of applications. During a deformable simulation, a nonlinear system needs to be repetitively solved in order to track the continuous shape evolution of the deforming body. A deformable object with complex geometry could house a large number of unknown degrees of freedom, and the resulting high-dimensional integration becomes prohibitive. To overcome this problem, this project will develop a re-branded deformable model which systematically integrates advanced simulation techniques and deep learning (DL) tools, specifically deep neural networks (DNNs). The hypothesis is that digital simulation provides us nearly unlimited noise-free training data, which should be fully exploited and leveraged to benefit unseen yet difficult simulation or computing challenges. Unlike existing data-driven methods that interpret the data with a closed-form formulation (e.g., using a convex interpolation), DNNs provide a universal mechanism to extract intrinsic features hidden behind the raw data in an end-to-end manner, and have already demonstrated significant outcomes in many long-standing computer vision problems like object detection, classification, and annotation. However, harnessing DL in physics-based simulation is not easy. While in theory one may still encode all of these parameters using a very high-dimensional input vector, the corresponding network would be extremely large and complex. Even if we manage to collect sufficient training data to optimize this net, a single forward pass of it may be slower than a conventional simulator, making DL completely unprofitable. In this project, we will thoroughly investigate those grand technical challenges, forge a collection of data structures and algorithmic techniques for the data-driven deformable simulation, and thereby pave the way for DL-based physics simulation to next-generation computer graphics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

世界上的一切都会变形，因此高质量变形建模成为严肃且现实驱动的视觉应用的核心算法成分，例如高保真动画、虚拟现实、医疗数据分析、手术模拟和数字制造/原型制作等。虽然变形的研究已经有几十年了，但变形模拟因其昂贵的计算而臭名昭著。随着复杂传感设备和采集技术的快速发展，数据的复杂性、规模和维度呈指数级增长，大规模几何图形在现代 3D 数据处理中变得无处不在。即使使用最先进的硬件，大规模的变形模拟仍然可能需要数小时、数天甚至数周的时间。在这个数据爆炸的时代，对计算效率和模拟真实性的要求不断提高，对这一经典计算问题提出了前所未有的挑战，因此需要针对大规模、复杂和非线性变形模型的改变游戏规则的算法技术来赋能未来的图形应用。 如果成功，该项目不仅将扩展基于物理的模拟技术的前沿，还将深刻启发图形之外的更广泛的计算社区，并实现各种应用。在变形模拟过程中，需要重复求解非线性系统，以跟踪变形体的连续形状演化。具有复杂几何形状的可变形物体可以容纳大量未知的自由度，并且由此产生的高维积分变得令人望而却步。为了克服这个问题，该项目将开发一种重新命名的变形模型，该模型系统地集成了先进的模拟技术和深度学习（DL）工具，特别是深度神经网络（DNN）。我们的假设是，数字仿真为我们提供了几乎无限的无噪声训练数据，应该充分利用和利用这些数据来应对看不见但困难的仿真或计算挑战。与使用封闭形式公式（例如使用凸插值）解释数据的现有数据驱动方法不同，DNN 提供了一种通用机制，以端到端的方式提取隐藏在原始数据背后的内在特征，并且已经在许多长期存在的计算机视觉问题（例如对象检测、分类和注释）中展示了显着的成果。然而，在基于物理的模拟中利用深度学习并不容易。虽然理论上人们仍然可以使用非常高维的输入向量对所有这些参数进行编码，但相应的网络将非常庞大和复杂。即使我们设法收集足够的训练数据来优化该网络，它的单次前向传播也可能比传统模拟器慢，从而使深度学习完全无利可图。在这个项目中，我们将深入研究这些重大技术挑战，为数据驱动的变形模拟建立一套数据结构和算法技术，从而为基于深度学习的物理模拟到下一代计算机图形学铺平道路。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。