III:Medium:Physics-guided Machine Learning for Predicting Cell Trajectories, Shapes, and Interactions in Complex Dynamic Environments

随着深度学习的进步不断革新计算机视觉领域，现在机器学习方法可以在行人和车辆跟踪等基准问题中预测移动物体的未来轨迹和行为。尽管有这些发展，目前用于预测物体未来轨迹的深度学习标准大多假设背景是静态的，物体的形状对运动是不变的。然而，在许多现实世界的应用中，我们经常遇到背景环境不断改变其结构的问题，这反过来直接影响到运动物体的形状、外观和未来轨迹的变化。例如，在机械生物学领域——研究活细胞运动的领域——当细胞在人体纤维环境中移动时，它们的形状、大小和轨迹都会发生巨大的变化，在这个过程中不断地拉动或推动背景纤维，重塑背景环境。该项目旨在开发新的机器学习方法，利用显微镜成像数据和细胞对背景环境施加的力的物理科学知识来研究细胞形状变化与背景环境之间的相互作用。我们的最终目标是发现不同背景配置下细胞行为的规则，并利用这些规则来预测细胞在许多科学和社会相关应用中的未来运动，如胚胎发育、伤口愈合、免疫反应和癌症转移的研究。人工智能的长期目标之一是教会机器如何预测未来。随着深度学习的进步，机器学习（ML）框架现在可以在几个计算机视觉应用中进行预测。我们提出了一个问题：深度学习方法能否提取出动态“变形”物体的运动规则——这些物体不断地根据环境调整其外观——并使用这些规则来预测它们未来的行为？我们在机械生物学的动机应用背景下研究这个问题，以预测和解释细胞如何移动，相互作用，重塑其环境，并根据我们体内不断变化的生理环境适应其外观。尽管深度学习在预测人体运动和车辆轨迹方面取得了成功，但这些方法在预测复杂现实环境中细胞运动动态的能力方面仍然存在根本性的差距。这主要是由于细胞形状的高度动态特性，当它们在运动过程中感知和反应环境时，它们会经历无限的转变。此外，细胞运动的动力学受到细胞对背景环境施加的力的物理特性以及细胞-细胞相互作用的复杂性的限制。该项目的愿景是开发一种新的物理引导机器学习（PGML）框架，以预测动态物理环境中变形物体的运动。我们的框架充分利用了“融合研究”的原则，通过整合来自三个不同学科的数据、知识和方法：机器学习、实验细胞成像和计算建模。我们项目的最终目标是通过分析我们的PGML框架在机械生物学背景下产生的可解释理论，催化发现新的“细胞行为规则”。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As advances in deep learning continue to revolutionize the field of computer vision, it is now possible for machine learning methods to predict the future trajectory and behavior of moving objects in benchmark problems such as pedestrian and vehicle tracking. Despite these developments, current standards in deep learning for predicting future trajectories of objects mostly assume the background to be static and the shapes of the objects to be invariant to motion. However, in many real-world applications, we routinely encounter problems where the background environment is constantly changing its structure, which in-turn directly affects changes in shape, appearance, and future trajectory of the moving objects. For example, in the area of mechanobiology—the field of study of movements of living cells—cells undergo massive transformations in their shape, size, and trajectory as they move across fibrous environments in the human body, continuously tugging or pushing on the background fibers and remodeling the background environment in the process. This project aims to develop novel machine learning methods to study the interplay between changes in cell shapes and background environments using microscopy imaging data and scientific knowledge of the physics of forces exerted by the cells on the background environments. Our ultimate objective is to discover the rules of cell behavior under varying background configurations and use these rules to predict future movements of cells in a number of scientific and societally relevant applications such as the study of embryo development, wound closure, immune response, and cancer metastasis. One of the long-standing goals of artificial intelligence has been to teach machines how to predict or forecast the future. With advances in deep learning, it is now possible for machine learning (ML) frameworks to make predictions in several computer vision applications. We ask the question: can deep learning methods extract the rules of motion of dynamic “shape-shifting” objects—that are constantly adapting their appearance in relation to their environment—and use these rules to predict their future behavior? We investigate this question in the context of a motivation application in mechanobiology to predict and explain how cells move, interact with each other, remodel their environment, and adapt their appearance with changing physiological environments inside our body. Despite the success of deep learning in predicting human motion and vehicle trajectories, fundamental gaps remain in the ability of these methods to predict the dynamics of cell motion in complex realistic environments. This is primarily due to the highly dynamic nature of cell shapes that undergo limitless transformations as they sense and react to their environment during motion. In addition, the dynamics of cell motion is constrained by the physics of forces exerted by the cells on the background environment, as well as the complex nature of cell-cell interactions. The vision of this project is to develop a novel physics-guided machine learning (PGML) framework to predict the motion of shape-shifting objects in dynamic physical environments. Our framework fully leverages the principles of “convergence research” by integrating data, knowledge, and methodologies from three different disciplines: machine learning, experimental cell imaging, and computational modeling. The ultimate goal of our project is to catalyze the discovery of new “rules of cell behavior” by analyzing explainable theories produced by our PGML framework in the context of mechanobiology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.