AF: Small: Collaborative Research: Distributed Quasi-Newton Methods for Nonsmooth Optimization

优化，即找到产生最小输出的数学函数的输入，是智能设备或云应用程序中许多进步背后的主力算法。随着数据变得越来越大，越来越分散，需要新的想法来保持优化的速度和准确性。算子分裂是一种新的一阶优化方法，它将最小化函数表示为两个凸函数的和，其中一个凸函数是光滑的，另一个凸函数是不可微的。本课题利用二阶优化方法探索算子分裂，该方法收敛到最小值的速度更快。重点是大型，分布式和流数据集，因此产生的通用数值求解器和嵌入式系统实现可以支持网络物理系统和物联网的优化。该项目优先考虑学生和研究人员的积极参与和培训，特别强调纳入妇女和代表性不足的少数群体。该项目不仅涉及美国三所顶尖大学的合作，还涉及欧洲研究机构KU Leuven的合作。具体而言，本研究项目试图将现有的结构化凸优化方法（如著名的ADMM算法）解释为应用于原始问题公式中产生的特定函数的梯度方法，并将算子分割技术解释为适当选择算子的不动点迭代。一个关键的理论基础是引入新的包络函数（具有相同解集的光滑上近似），它可以用作可变度量回溯线搜索的优点函数。总而言之，该项目的主要重点是设计适用于涉及大数据并对决策施加严格实时约束的大规模多智能体网络物理系统的分布式异步方法。在这个范围内，目标是提供在以下方面优于当前最先进的方法：(a)计算速度，(b)大数据规模的可扩展性，(c)对各种不确定性的鲁棒性，以及最重要的(d)实时网络上的分布式异步实现。其优点将在信号处理、控制、机器学习和机器人技术的应用背景下加以说明。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Optimization, which finds the inputs to a mathematical function that produce the minimum output, is a workhorse algorithm behind many of the advances in smart devices or applications in the cloud. As data gets larger and more distributed, new ideas are needed to maintain the speed and accuracy of optimization. Operator splitting, which expresses the function to minimize as the sum of two convex functions, one of which is smooth and the other non-differentiable, is an idea that has produced to new first-order optimization methods. This project explores operator splitting with second-order optimization methods, which have faster convergence to the minimum. The focus is on large, distributed, and streaming data sets, so that the resulting general-purpose numerical solvers and embedded systems implementations can support optimization in cyberphysical systems and the Internet-of-Things. The project has as priority the active engagement and training of students and researchers, with specific emphasis on the inclusion of women and under-represented minority groups. This project not only involves collaboration across three top-tier American universities, but also with European research institute, KU Leuven. In specific, this research project seeks to interpret existing methods for structured convex optimization (such as the celebrated ADMM algorithm) as gradient methods applied to specific functions arising from the original problem formulation, and interpret of operator-splitting techniques as fixed point iterations for appropriately selected operators. A key theoretical foundation is the introduction of new envelope functions (smooth upper approximations possessing the same sets of solutions) that can be used as merit functions for variable-metric backtracking line-search. To conclude, a principal focus of the project is to design distributed asynchronous methods applicable to large-scale multi-agent cyberphysical systems that involve big data and impose stringent real-time constraints for decision-making. In this purview, the goal is to deliver methods that will outperform current state-of-the-art in terms of (a) speed of computations, (b) scalability with big data sizes, (c) robustness to various types of uncertainty, and, most topically, (d) distributed asynchronous implementation over networks in real-time. The merits will be illustrated in the context of applications in signal processing, control, machine learning and robotics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.