III: Small: Collaborative Research: Learning Active Physics-Based Models from Data

该项目探索了一种新的算法框架，用于自动生成自然世界中物体的数字模型，忠实地再现其物理对应物的结构和功能。我们特别专注于建模活动的可变形对象，即，能够在自身体内产生内力的物体，如生物肌肉或机器人执行器。我们的方法与传统的建模管道不同，它是从实际机制的示例数据中学习数字模型，而不是从第一原理手动设计它们。我们将当前最先进的深度学习技术应用于我们的问题，特别是人工神经网络，通过赋予它们关于可变形材料的物理行为的知识。预计这将显著提升通用神经网络的能力，否则它将被迫从数据中学习物理定律，这是一项不必要的任务，因为可变形介质的基本属性，如能量守恒和旋转不变性，应该被视为理所当然。所提出的算法框架将大大简化我们自然世界中物体的数字复制品的创建，同时提高它们的保真度。这将使虚拟和增强现实部署能够在教育和技能培训应用中提供逼真的体验，例如虚拟手术室或紧急响应场景。计算机托管的功能对象的双打也是一个有价值的原型工具，在设计和优化的物理功能副本，如假肢devices.To实现这些目标，我们杂交的神经网络与微分模拟器，输出的准静态（即平衡）的形状作为输入控制参数的函数的主动弹性模型，并规定（已知）的边界条件。基于有限元的模拟器基于投影动力学，设计时考虑到了可微性，这是一个关键特性，可以与经典的反向传播算法顺利结合，并集成到现有的深度学习框架中，如PyTorch。模拟器的输入允许以非常精细的粒度规定致动控制，从而潜在地使每个有限元成为其自己的独立可控致动器。这些细粒度的致动控制将由卷积神经网络生成，卷积神经网络使用低维时变控制向量和常数（即，时不变）网络权重。我们训练这个聚合管道，联合推断控制网络的权重以及与不同输入配置相关的潜变量的值，以便最好地将训练集解释为低维控制空间的动作。该核心框架随后将扩展到1）允许处理接触和碰撞，2）优化空间变化的材料参数，3）提升准静态假设并模拟时变动态。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project explores a novel algorithmic framework for automatic generation of digital models of objects from our natural world, that faithfully reproduce the structure and function of their physical counterparts. We specifically focus on modeling active deformable objects, i.e., objects capable of producing internal forces within their own bodies, such as biological muscles or robotic actuators. Our approach differs from the traditional modeling pipeline by learning the digital models from example data of the mechanism in-action, rather than by manually engineering them from the first principles. We adapt current state-of-the-art deep learning techniques to our problem, in particular artificial neural networks, by endowing them with knowledge about the physics-based behavior of deformable materials. This is expected to significantly upgrade the capabilities of generic neural networks, which would be otherwise forced to learn the laws of physics from data, which is an unnecessary task because fundamental properties of deformable media, such as conservation of energy and rotational invariance, should simply be taken for granted. The proposed algorithmic framework will greatly simplify the creation of digital replicas of objects in our natural world, while enhancing their fidelity. This will empower Virtual and Augmented Reality deployments to deliver life-like experiences in educational and skill-training applications, such as virtual operating rooms or emergency response scenarios. Computer-hosted doubles of functional objects are also a valuable prototyping tool in the design and optimization of physical functional replicas, such as prosthetic devices.To achieve these goals, we hybridize a neural network with a differentiable simulator, which outputs the quasistatic (i.e. equilibrated) shape of an active elastic model as a function of input control parameters, and subject to prescribed (known) boundary conditions. The finite element-based simulator is based on Projective Dynamics and designed with differentiability in mind, which is a key feature that will enable smooth combination with the classical backpropagation algorithm and integration within existing deep learning frameworks, such as PyTorch. The input to the simulator allows the actuation controls to be prescribed at very fine granularity, potentially enabling each finite element to become its own independently controllable actuator. These fine-grained actuation controls will be generated by a convolutional neural network, which creates them using a low-dimensional time-varying control vector and constant (i.e., time-invariant) network weights. We train this aggregate pipeline, jointly inferring both the weights of the control network as well as the values of the latent variables associated with different input configurations, as to best explain the training set as the action of a low-dimensional control space. This core framework will subsequently be extended to 1) allow for processing of contact and collisions, 2) optimization of spatially-varying material parameters, 3) lifting the quasi-statics assumption and simulating time-varying dynamics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.