CIF: Medium: Multi-Agent Consensus Equilibrium: Modular Methods for Integrating Disparate Sources of Expertise

在过去十年中，有两大趋势重塑了数据科学：(i)成像和传感设备的浪潮日益增长；（ii）突破性机器学习技术的迅速采用。从自动驾驶汽车到医疗设备，收集和分析大量数据的能力正在改变世界。本研究项目的目标是创建一个新的数学和计算框架，将分布式和多模态传感器数据集成到数据科学和机器学习中多种类型的新兴模型中，从而从数据中提取更多信息。项目团队包括来自不同学科的研究人员，他们将解决从物理科学和医学到消费者成像和工业检测等问题。该研究将产生理论、算法和开放软件，可用于整合来自异构传感系统的信息，以估计和重建信号和图像。模型数据集成的框架是基于一个新的多智能体共识均衡理论。MACE允许将多模态物理传感器信息与来自数据科学模型的信息进行模块化集成。该方法的核心是一致平衡方程的计算解。这些方程平衡了分布式传感器信息和机器学习模型提供的先验知识。MACE框架是更传统的贝叶斯或正则化逆方法的推广，但它允许在传感和成像问题的解决方案中使用非传统的数据科学模型，如深度卷积神经网络。本项目的贡献主要体现在四个方面：第一，基础理论方法；重点2：鲁棒传感器与数据模型集成；推力3：多模态网络化MACE；推力4：自动化实验。该项目还包括综合教育活动，吸引研究生和本科生参与这项研究，以及开发关于共识平衡和非线性光学成像的新课程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Over the past decade, two major trends have reshaped data science: (i) a growing tidal wave of imaging and sensing devices and (ii) the rapid adoption of break-through machine learning technologies. From autonomous vehicles to medical devices, the ability to collect and analyze large quantities of data is changing the world. The goal of this research project is to create a new mathematical and computational framework for integrating distributed and multimodal sensor data into multiple types of emerging models in data science and machine learning so as to extract more information from data. The project team includes researchers from diverse disciplines who will address problems ranging from physical science and medicine to consumer imaging and industrial inspection. The research will result in theories, algorithms, and open software that can be used to integrate information from heterogeneous sensing systems to estimate and reconstruct signals and images.The framework for model-data integration is based on a new theory of Multi-Agent Consensus Equilibrium (MACE). MACE allows for modular integration of multi-modal physical sensor information with information derived from data science models. At the core of this approach is the computational solution of the consensus equilibrium equations. These equations balance distributed sensor information with prior knowledge provided by machine learning models. The MACE framework is a generalization of the more traditional Bayesian or regularized inverse approach, but it allows for the use of non-traditional data science models such as deep convolutional neural networks in the solution of sensing and imaging problems. This project's contributions are in four areas: Thrust 1: Foundational Theoretical Methods; Thrust 2: Robust Sensor and Data Model Integration; Thrust 3: Multimodal and Networked MACE; and Thrust 4: Automated Experimentation. The project also includes integrated educational activities, engaging both graduate and undergraduate students in this research, as well as the development of new courses on consensus equilibrium and nonlinear optical imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.