III:Medium:Physics-guided Machine Learning for Predicting Cell Trajectories, Shapes, and Interactions in Complex Dynamic Environments

III：中：物理引导机器学习，用于预测复杂动态环境中的细胞轨迹、形状和相互作用

• 批准号: 2107332
• 负责人: Anuj Karpatne
• 金额: $ 100万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107332&HistoricalAwards=false
• 关键词: III; Medium; Physics; guided; Machine

As advances in deep learning continue to revolutionize the field of computer vision, it is now possible for machine learning methods to predict the future trajectory and behavior of moving objects in benchmark problems such as pedestrian and vehicle tracking. Despite these developments, current standards in deep learning for predicting future trajectories of objects mostly assume the background to be static and the shapes of the objects to be invariant to motion. However, in many real-world applications, we routinely encounter problems where the background environment is constantly changing its structure, which in-turn directly affects changes in shape, appearance, and future trajectory of the moving objects. For example, in the area of mechanobiology—the field of study of movements of living cells—cells undergo massive transformations in their shape, size, and trajectory as they move across fibrous environments in the human body, continuously tugging or pushing on the background fibers and remodeling the background environment in the process. This project aims to develop novel machine learning methods to study the interplay between changes in cell shapes and background environments using microscopy imaging data and scientific knowledge of the physics of forces exerted by the cells on the background environments. Our ultimate objective is to discover the rules of cell behavior under varying background configurations and use these rules to predict future movements of cells in a number of scientific and societally relevant applications such as the study of embryo development, wound closure, immune response, and cancer metastasis. One of the long-standing goals of artificial intelligence has been to teach machines how to predict or forecast the future. With advances in deep learning, it is now possible for machine learning (ML) frameworks to make predictions in several computer vision applications. We ask the question: can deep learning methods extract the rules of motion of dynamic “shape-shifting” objects—that are constantly adapting their appearance in relation to their environment—and use these rules to predict their future behavior? We investigate this question in the context of a motivation application in mechanobiology to predict and explain how cells move, interact with each other, remodel their environment, and adapt their appearance with changing physiological environments inside our body. Despite the success of deep learning in predicting human motion and vehicle trajectories, fundamental gaps remain in the ability of these methods to predict the dynamics of cell motion in complex realistic environments. This is primarily due to the highly dynamic nature of cell shapes that undergo limitless transformations as they sense and react to their environment during motion. In addition, the dynamics of cell motion is constrained by the physics of forces exerted by the cells on the background environment, as well as the complex nature of cell-cell interactions. The vision of this project is to develop a novel physics-guided machine learning (PGML) framework to predict the motion of shape-shifting objects in dynamic physical environments. Our framework fully leverages the principles of “convergence research” by integrating data, knowledge, and methodologies from three different disciplines: machine learning, experimental cell imaging, and computational modeling. The ultimate goal of our project is to catalyze the discovery of new “rules of cell behavior” by analyzing explainable theories produced by our PGML framework in the context of mechanobiology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着深度学习的进步继续彻底改变计算机视觉领域，机器学习方法现在可以在行人和车辆跟踪等基准问题中预测移动对象的未来轨迹和行为。尽管有这些发展，目前用于预测对象未来轨迹的深度学习标准大多假设背景是静态的，对象的形状对运动是不变的。然而，在许多现实世界的应用中，我们经常会遇到背景环境不断改变其结构的问题，这反过来又直接影响移动对象的形状，外观和未来轨迹的变化。例如，在研究活细胞运动的机械生物学领域，当细胞在人体内的纤维环境中移动时，它们的形状、大小和轨迹会发生巨大的变化，在此过程中，不断地牵引或推动背景纤维并重塑背景环境。该项目旨在开发新的机器学习方法，利用显微镜成像数据和细胞对背景环境施加的力的物理学科学知识，研究细胞形状和背景环境变化之间的相互作用。我们的最终目标是发现不同背景配置下细胞行为的规则，并使用这些规则来预测细胞在许多科学和社会相关应用中的未来运动，例如胚胎发育，伤口闭合，免疫反应和癌症转移的研究。人工智能的一个长期目标是教机器如何预测或预测未来。随着深度学习的进步，机器学习（ML）框架现在可以在几个计算机视觉应用中进行预测。我们提出这样一个问题：深度学习方法是否可以提取动态“变形”对象的运动规则--这些对象不断地根据环境来调整它们的外观--并使用这些规则来预测它们未来的行为？我们在机械生物学中的动机应用的背景下研究这个问题，以预测和解释细胞如何移动，相互作用，重塑它们的环境，并适应它们的外观与我们体内不断变化的生理环境。尽管深度学习在预测人体运动和车辆轨迹方面取得了成功，但这些方法在预测复杂现实环境中细胞运动动力学的能力方面仍然存在根本差距。这主要是由于细胞形状的高度动态性质，当它们在运动过程中感知并对环境做出反应时，它们会经历无限的变化。此外，细胞运动的动力学受到细胞对背景环境施加的力的物理学以及细胞-细胞相互作用的复杂性质的约束。该项目的愿景是开发一种新的物理引导机器学习（PGML）框架，以预测动态物理环境中形状变化对象的运动。我们的框架通过整合来自三个不同学科的数据，知识和方法充分利用了“融合研究”的原则：机器学习，实验细胞成像和计算建模。该项目的最终目标是通过分析我们的PGML框架在机械生物学背景下产生的可解释理论，促进新的“细胞行为规则”的发现。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

• DOI: 10.48550/arxiv.2310.09441
• 发表时间: 2023-10
• 期刊: ArXiv
• 作者: Medha Sawhney;Bhas Karmarkar;E. Leaman;Arka Daw;A. Karpatne;B. Behkam

Ultra-thin and ultra-porous nanofiber networks as a basement-membrane mimic

• DOI: 10.1039/d3lc00304c
• 发表时间: 2023-09-19
• 期刊: LAB ON A CHIP
• 影响因子: 6.1
• 作者: Graybill，Philip M.;Jacobs，Edward J.;Davalos，Rafael V.
• 通讯作者: Davalos，Rafael V.

Mitigating Propagation Failures in Physics-Informed Neural Networks Using Retain-Resample-Release (R3) Sampling

使用保留-重采样-释放 (R3) 采样减轻物理信息神经网络中的传播失败

• 发表时间: 2023
• 期刊: Proceedings of the 40th International Conference on Machine Learning
• 作者: Daw, Arka;Bu, Jie;Wang, Sifan;Perdikaris, Paris;Karpatne, Anuj
• 通讯作者: Karpatne, Anuj

Detecting and Tracking Hard-to-Detect Bacteria in Dense Porous Backgrounds

检测和追踪致密多孔背景中难以检测的细菌

• 发表时间: 2023
• 期刊: CVPR Workshop on CV4Animals 2023
• 作者: Sawhney, Medha;Karmarkar, Bhas;Leaman, Eric J.;Daw, Arka;Karpatne, Anuj;Behkam, Bahareh
• 通讯作者: Behkam, Bahareh

Deep Learning Enabled Label-free Cell Force Computation in Deformable Fibrous Environments

深度学习在可变形纤维环境中实现无标记细胞力计算

• 发表时间: 2022
• 期刊: bioRxiv
• 作者: Abinash Padhi, Arka Daw