III:Medium:Physics-guided Machine Learning for Predicting Cell Trajectories, Shapes, and Interactions in Complex Dynamic Environments

III：中：物理引导机器学习，用于预测复杂动态环境中的细胞轨迹、形状和相互作用

基本信息

批准号：
2107332
负责人：
Anuj Karpatne
金额：
$ 100万
依托单位：
Virginia Polytechnic Institute and State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-10-01 至 2025-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107332&HistoricalAwards=false
关键词：
III Medium Physics guided Machine

项目摘要

As advances in deep learning continue to revolutionize the field of computer vision, it is now possible for machine learning methods to predict the future trajectory and behavior of moving objects in benchmark problems such as pedestrian and vehicle tracking. Despite these developments, current standards in deep learning for predicting future trajectories of objects mostly assume the background to be static and the shapes of the objects to be invariant to motion. However, in many real-world applications, we routinely encounter problems where the background environment is constantly changing its structure, which in-turn directly affects changes in shape, appearance, and future trajectory of the moving objects. For example, in the area of mechanobiology—the field of study of movements of living cells—cells undergo massive transformations in their shape, size, and trajectory as they move across fibrous environments in the human body, continuously tugging or pushing on the background fibers and remodeling the background environment in the process. This project aims to develop novel machine learning methods to study the interplay between changes in cell shapes and background environments using microscopy imaging data and scientific knowledge of the physics of forces exerted by the cells on the background environments. Our ultimate objective is to discover the rules of cell behavior under varying background configurations and use these rules to predict future movements of cells in a number of scientific and societally relevant applications such as the study of embryo development, wound closure, immune response, and cancer metastasis. One of the long-standing goals of artificial intelligence has been to teach machines how to predict or forecast the future. With advances in deep learning, it is now possible for machine learning (ML) frameworks to make predictions in several computer vision applications. We ask the question: can deep learning methods extract the rules of motion of dynamic “shape-shifting” objects—that are constantly adapting their appearance in relation to their environment—and use these rules to predict their future behavior? We investigate this question in the context of a motivation application in mechanobiology to predict and explain how cells move, interact with each other, remodel their environment, and adapt their appearance with changing physiological environments inside our body. Despite the success of deep learning in predicting human motion and vehicle trajectories, fundamental gaps remain in the ability of these methods to predict the dynamics of cell motion in complex realistic environments. This is primarily due to the highly dynamic nature of cell shapes that undergo limitless transformations as they sense and react to their environment during motion. In addition, the dynamics of cell motion is constrained by the physics of forces exerted by the cells on the background environment, as well as the complex nature of cell-cell interactions. The vision of this project is to develop a novel physics-guided machine learning (PGML) framework to predict the motion of shape-shifting objects in dynamic physical environments. Our framework fully leverages the principles of “convergence research” by integrating data, knowledge, and methodologies from three different disciplines: machine learning, experimental cell imaging, and computational modeling. The ultimate goal of our project is to catalyze the discovery of new “rules of cell behavior” by analyzing explainable theories produced by our PGML framework in the context of mechanobiology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着深度学习的进步继续彻底改变计算机视野的领域，机器学习方法现在可以预测基准问题（例如行人和车辆跟踪）中移动对象的未来轨迹和行为。尽管有这些发展，但深度学习的当前标准用于预测对象的未来轨迹，主要假设背景是静态的，并且对象的形状是运动不变的。但是，在许多现实世界中，我们通常会遇到背景环境不断改变其结构的问题，这直接影响移动对象的形状，外观和未来轨迹的变化。例如，在机制的领域（生物细胞运动的研究领域），当它们在人体的纤维环境中移动时，会经历其形状，大小和轨迹的大规模转变，不断地拖拉或推动背景纤维，并在此过程中重塑背景环境。该项目旨在开发新型的机器学习方法，以使用显微镜成像数据以及对单元在背景环境上执行的力的物理学的科学知识来研究细胞形状变化与背景环境之间的相互作用。我们的最终目标是在不同的背景配置下发现细胞行为的规则，并使用这些规则来预测许多科学和社会相关的应用中的细胞的未来运动，例如胚胎发育研究，伤口闭合，免疫反应，免疫反应和癌症转移。人工智能的长期目标之一是教机器如何预测或预测未来。随着深度学习的进步，机器学习（ML）框架现在可以在几个计算机视觉应用中做出预测。我们提出一个问题：深度学习方法可以提取动态“变形”对象的运动规则 - 不断调整其与环境相关的外观，并使用这些规则来预测其未来行为？我们在机制中的动机应用程序中进行了调查，以预测和解释细胞如何移动，相互作用，重塑其环境，并通过不断变化的身体环境来调整其外观。尽管深度学习在预测人类运动和车辆轨迹方面取得了成功，但这些方法在复杂逼真的环境中预测细胞运动动态的能力仍然存在。这主要是由于细胞形状的高度动态性质，在运动过程中感觉到并反应环境时会经历无限转换。另外，细胞运动的动力学受到细胞在背景环境上执行的力的物理以及细胞 - 细胞相互作用的复杂性质的约束。该项目的愿景是开发一种新颖的物理指导机器学习（PGML）框架，以预测动态物理环境中形状转移对象的运动。我们的框架通过整合来自三个不同学科的数据，知识和方法来充分利用“融合研究”的原理：机器学习，实验性单元格成像和计算建模。我们项目的最终目的是通过在机械学生物学的背景下分析我们的PGML框架所产生的可解释理论来催化新的“细胞行为规则”。该奖项反映了NSF的法定任务，并被认为是通过使用基金会的智力和更广泛影响的评估来审查Criteria来通过评估来通过评估来获得支持的。

项目成果

期刊论文数量（8）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

DOI：
10.48550/arxiv.2310.09441
发表时间：
2023-10
期刊：
ArXiv
影响因子：
0
作者：
Medha Sawhney;Bhas Karmarkar;E. Leaman;Arka Daw;A. Karpatne;B. Behkam
通讯作者：
Medha Sawhney;Bhas Karmarkar;E. Leaman;Arka Daw;A. Karpatne;B. Behkam

Ultra-thin and ultra-porous nanofiber networks as a basement-membrane mimic

DOI：
10.1039/d3lc00304c
发表时间：
2023-09-19
期刊：
LAB ON A CHIP
影响因子：
6.1
作者：
Graybill，Philip M.;Jacobs，Edward J.;Davalos，Rafael V.
通讯作者：
Davalos，Rafael V.

Mitigating Propagation Failures in Physics-Informed Neural Networks Using Retain-Resample-Release (R3) Sampling

使用保留-重采样-释放 (R3) 采样减轻物理信息神经网络中的传播失败

DOI：
发表时间：
2023
期刊：
Proceedings of the 40th International Conference on Machine Learning
影响因子：
0
作者：
Daw, Arka;Bu, Jie;Wang, Sifan;Perdikaris, Paris;Karpatne, Anuj
通讯作者：
Karpatne, Anuj

Deep Learning Enabled Label-free Cell Force Computation in Deformable Fibrous Environments

深度学习在可变形纤维环境中实现无标记细胞力计算

DOI：
发表时间：
2022
期刊：
bioRxiv
影响因子：
0
作者：
Abinash Padhi, Arka Daw
通讯作者：
Abinash Padhi, Arka Daw

Detecting and Tracking Hard-to-Detect Bacteria in Dense Porous Backgrounds

检测和追踪致密多孔背景中难以检测的细菌

DOI：
发表时间：
2023
期刊：
CVPR Workshop on CV4Animals 2023
影响因子：
0
作者：
Sawhney, Medha;Karmarkar, Bhas;Leaman, Eric J.;Daw, Arka;Karpatne, Anuj;Behkam, Bahareh
通讯作者：
Behkam, Bahareh