Incorporating physics into data-driven computer vision
Incorporating physics into data-driven computer vision
复制标题
DOI：
10.1038/s42256-023-00662-0
复制
发表时间：
2023-06
影响因子：
23.8
通讯作者：
A. Kadambi;Celso de Melo;Cho-Jui Hsieh;Mani Srivastava;Stefano Soatto
中科院分区：
文献类型：
--
作者：
A. Kadambi;Celso de Melo;Cho-Jui Hsieh;Mani Srivastava;Stefano Soatto
许多计算机视觉技术从图像中推断出我们物理世界的属性。尽管图像是通过光和力学的物理学形成的，但计算机视觉技术通常是数据驱动的。这种趋势主要与性能相关：与现代深度学习相比，基于物理视觉的经典技术在指标上的得分通常较低。然而，本视角中涵盖的最新研究表明，物理模型可以作为约束包含到数据驱动的管道中。这样做时，人们可以将数据驱动方法的性能优势与基于物理的方法提供的优势（例如不可解释性、可证伪性和概括性）结合起来。本视角的目的是概述将物理模型集成到人工智能管道中的具体方法，称为基于物理的机器学习。我们讨论的技术方法包括数据集修改、网络设计、损失函数、优化和正则化方案。
Many computer vision techniques infer properties of our physical world from images. Although images are formed through the physics of light and mechanics, computer vision techniques are typically data driven. This trend is mostly performance related: classical techniques from physics-based vision often score lower on metrics compared with modern deep learning. However, recent research, covered in this Perspective, has shown that physical models can be included as a constraint into data-driven pipelines. In doing so, one can combine the performance benefits of a data-driven method with advantages offered from a physics-based method, such as intepretability, falsifiability and generalizability. The aim of this Perspective is to provide an overview into specific approaches for integrating physical models into artificial intelligence pipelines, referred to as physics-based machine learning. We discuss technical approaches that range from modifications to the dataset, network design, loss functions, optimization and regularization schemes.