III: Small: Collaborative Research: Learning Active Physics-Based Models from Data

这个项目探索了一种新的算法框架，用于自动生成来自我们自然世界的物体的数字模型，忠实地复制它们的物理对应物的结构和功能。我们特别专注于对主动可变形对象进行建模，即能够在自己的身体内产生内力的对象，如生物肌肉或机器人执行器。我们的方法不同于传统的建模管道，通过从作用中的机构的示例数据学习数字模型，而不是从第一原理手动设计它们。我们将当前最先进的深度学习技术，特别是人工神经网络，通过赋予它们关于可变形材料的基于物理的行为的知识，来适应我们的问题。这预计将显著提升通用神经网络的能力，否则它将被迫从数据中学习物理定律，这是一项不必要的任务，因为可变形介质的基本属性，如能量守恒和旋转不变性，应该被视为理所当然。拟议的算法框架将极大地简化我们自然界中物体的数字复制品的创建，同时提高它们的保真度。这将使虚拟和增强现实部署能够在教育和技能培训应用程序(如虚拟手术室或紧急响应场景)中提供逼真的体验。计算机托管的双重功能对象也是设计和优化物理功能复制品(如假肢设备)的有价值的原型工具。为了实现这些目标，我们将神经网络与可微模拟器相结合，该模拟器输出作为输入控制参数的函数的主动弹性模型的准静态(即平衡)形状，并受规定(已知)边界条件的约束。基于有限元的模拟器以投影动力学为基础，并在设计时考虑了可微性，这是一个关键特性，将使其能够与经典的反向传播算法顺利结合，并集成到现有的深度学习框架中，如PyTorch。模拟器的输入允许以非常精细的粒度指定驱动控制，潜在地使每个有限元成为其自己的独立可控的执行器。这些细粒度的驱动控制将由卷积神经网络生成，该网络使用低维时变控制向量和恒定(即，时间不变)的网络权重来创建它们。我们训练这个聚合管道，联合推断控制网络的权重以及与不同输入配置相关联的潜在变量的值，以便最好地将训练集解释为低维控制空间的作用。这一核心框架随后将扩展到1)允许处理接触和碰撞，2)优化空间变化的材料参数，3)取消准静态假设并模拟时变动态。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project explores a novel algorithmic framework for automatic generation of digital models of objects from our natural world, that faithfully reproduce the structure and function of their physical counterparts. We specifically focus on modeling active deformable objects, i.e., objects capable of producing internal forces within their own bodies, such as biological muscles or robotic actuators. Our approach differs from the traditional modeling pipeline by learning the digital models from example data of the mechanism in-action, rather than by manually engineering them from the first principles. We adapt current state-of-the-art deep learning techniques to our problem, in particular artificial neural networks, by endowing them with knowledge about the physics-based behavior of deformable materials. This is expected to significantly upgrade the capabilities of generic neural networks, which would be otherwise forced to learn the laws of physics from data, which is an unnecessary task because fundamental properties of deformable media, such as conservation of energy and rotational invariance, should simply be taken for granted. The proposed algorithmic framework will greatly simplify the creation of digital replicas of objects in our natural world, while enhancing their fidelity. This will empower Virtual and Augmented Reality deployments to deliver life-like experiences in educational and skill-training applications, such as virtual operating rooms or emergency response scenarios. Computer-hosted doubles of functional objects are also a valuable prototyping tool in the design and optimization of physical functional replicas, such as prosthetic devices.To achieve these goals, we hybridize a neural network with a differentiable simulator, which outputs the quasistatic (i.e. equilibrated) shape of an active elastic model as a function of input control parameters, and subject to prescribed (known) boundary conditions. The finite element-based simulator is based on Projective Dynamics and designed with differentiability in mind, which is a key feature that will enable smooth combination with the classical backpropagation algorithm and integration within existing deep learning frameworks, such as PyTorch. The input to the simulator allows the actuation controls to be prescribed at very fine granularity, potentially enabling each finite element to become its own independently controllable actuator. These fine-grained actuation controls will be generated by a convolutional neural network, which creates them using a low-dimensional time-varying control vector and constant (i.e., time-invariant) network weights. We train this aggregate pipeline, jointly inferring both the weights of the control network as well as the values of the latent variables associated with different input configurations, as to best explain the training set as the action of a low-dimensional control space. This core framework will subsequently be extended to 1) allow for processing of contact and collisions, 2) optimization of spatially-varying material parameters, 3) lifting the quasi-statics assumption and simulating time-varying dynamics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.