III:Medium:Physics-guided Machine Learning for Predicting Cell Trajectories, Shapes, and Interactions in Complex Dynamic Environments

随着深度学习的进步继续给计算机视觉领域带来革命性的变化，机器学习方法现在有可能在行人和车辆跟踪等基准问题中预测未来运动对象的轨迹和行为。尽管有这些发展，但目前用于预测对象未来轨迹的深度学习标准大多假设背景是静态的，并且对象的形状不随运动而变化。然而，在许多实际应用中，我们经常会遇到背景环境不断变化其结构的问题，这反过来又直接影响到运动对象的形状、外观和未来轨迹的变化。例如，在机械生物学领域--研究活细胞运动的领域--细胞在穿过人体内的纤维环境时，在形状、大小和轨迹上经历了巨大的变化，在这个过程中不断地拖拽或推动背景纤维，并重塑背景环境。该项目旨在开发新的机器学习方法，利用显微成像数据和细胞对背景环境施加的力的物理物理知识来研究细胞形状变化和背景环境之间的相互作用。我们的最终目标是发现细胞在不同背景配置下的行为规律，并使用这些规律来预测细胞在一些科学和社会相关应用中的未来运动，例如胚胎发育、伤口闭合、免疫反应和癌症转移的研究。人工智能的长期目标之一是教会机器如何预测或预测未来。随着深度学习的进步，机器学习(ML)框架现在有可能在几个计算机视觉应用中做出预测。我们提出了这样一个问题：深度学习方法能否提取动态“变形”物体的运动规则，并使用这些规则来预测它们未来的行为？我们在机械生物学的动机应用的背景下研究这个问题，以预测和解释细胞如何移动，相互作用，重塑它们的环境，并根据我们体内不断变化的生理环境来调整它们的外观。尽管深度学习在预测人体运动和车辆轨迹方面取得了成功，但这些方法在预测复杂现实环境中细胞运动动力学的能力方面仍然存在根本差距。这主要是由于细胞形状的高度动态性质，当它们在运动过程中感知环境并对环境做出反应时，它们会经历无限的变化。此外，细胞运动的动力学受到细胞对背景环境施加的力的物理性质以及细胞-细胞相互作用的复杂性质的限制。该项目的愿景是开发一种新的物理制导的机器学习(PGML)框架来预测动态物理环境中变形对象的运动。我们的框架通过整合来自三个不同学科的数据、知识和方法，充分利用了“融合研究”的原则：机器学习、实验细胞成像和计算建模。我们项目的最终目标是通过在机械生物学的背景下分析我们的PGML框架产生的可解释的理论来催化新的“细胞行为规则”的发现。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As advances in deep learning continue to revolutionize the field of computer vision, it is now possible for machine learning methods to predict the future trajectory and behavior of moving objects in benchmark problems such as pedestrian and vehicle tracking. Despite these developments, current standards in deep learning for predicting future trajectories of objects mostly assume the background to be static and the shapes of the objects to be invariant to motion. However, in many real-world applications, we routinely encounter problems where the background environment is constantly changing its structure, which in-turn directly affects changes in shape, appearance, and future trajectory of the moving objects. For example, in the area of mechanobiology—the field of study of movements of living cells—cells undergo massive transformations in their shape, size, and trajectory as they move across fibrous environments in the human body, continuously tugging or pushing on the background fibers and remodeling the background environment in the process. This project aims to develop novel machine learning methods to study the interplay between changes in cell shapes and background environments using microscopy imaging data and scientific knowledge of the physics of forces exerted by the cells on the background environments. Our ultimate objective is to discover the rules of cell behavior under varying background configurations and use these rules to predict future movements of cells in a number of scientific and societally relevant applications such as the study of embryo development, wound closure, immune response, and cancer metastasis. One of the long-standing goals of artificial intelligence has been to teach machines how to predict or forecast the future. With advances in deep learning, it is now possible for machine learning (ML) frameworks to make predictions in several computer vision applications. We ask the question: can deep learning methods extract the rules of motion of dynamic “shape-shifting” objects—that are constantly adapting their appearance in relation to their environment—and use these rules to predict their future behavior? We investigate this question in the context of a motivation application in mechanobiology to predict and explain how cells move, interact with each other, remodel their environment, and adapt their appearance with changing physiological environments inside our body. Despite the success of deep learning in predicting human motion and vehicle trajectories, fundamental gaps remain in the ability of these methods to predict the dynamics of cell motion in complex realistic environments. This is primarily due to the highly dynamic nature of cell shapes that undergo limitless transformations as they sense and react to their environment during motion. In addition, the dynamics of cell motion is constrained by the physics of forces exerted by the cells on the background environment, as well as the complex nature of cell-cell interactions. The vision of this project is to develop a novel physics-guided machine learning (PGML) framework to predict the motion of shape-shifting objects in dynamic physical environments. Our framework fully leverages the principles of “convergence research” by integrating data, knowledge, and methodologies from three different disciplines: machine learning, experimental cell imaging, and computational modeling. The ultimate goal of our project is to catalyze the discovery of new “rules of cell behavior” by analyzing explainable theories produced by our PGML framework in the context of mechanobiology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.