AF: Small: Collaborative Research: Distributed Quasi-Newton Methods for Nonsmooth Optimization

优化，即找到一个数学函数的输入，从而产生最小的输出，是云中智能设备或应用程序许多进步背后的主力算法。随着数据变得更大、更分散，需要新的想法来保持优化的速度和准确性。算子分裂是一种新的一阶优化方法，它把要最小化的函数表示为两个凸函数之和，其中一个凸函数是光滑的，另一个是不可微的。这个项目探索了二阶优化方法的算子分裂，这种方法具有更快的收敛速度到最小。重点放在大型、分布式和流数据集上，以便生成的通用数值求解器和嵌入式系统实现可以支持网络物理系统和物联网中的优化。该项目将学生和研究人员的积极参与和培训作为优先事项，特别强调吸纳妇女和代表性不足的少数群体。这个项目不仅涉及美国三所一流大学的合作，还包括与欧洲鲁汶大学研究机构的合作。具体地说，本研究项目试图将现有的结构化凸优化方法(如著名的ADMM算法)解释为应用于原始问题公式中的特定函数的梯度方法，并将算子分裂技术解释为适当选择的算子的不动点迭代。一个关键的理论基础是引入了新的包络函数(具有相同解集的光滑上近似)，这些包络函数可以用作变尺度回溯直线搜索的价值函数。总而言之，该项目的一个主要重点是设计适用于大规模多代理网络物理系统的分布式异步方法，这些系统涉及大数据并对决策施加严格的实时约束。在这个范围内，目标是提供在以下方面优于当前最先进水平的方法：(A)计算速度，(B)大数据量的可扩展性，(C)对各种类型的不确定性的稳健性，以及(D)通过网络实时的分布式异步实现。该奖项将在信号处理、控制、机器学习和机器人应用的背景下进行说明。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Optimization, which finds the inputs to a mathematical function that produce the minimum output, is a workhorse algorithm behind many of the advances in smart devices or applications in the cloud. As data gets larger and more distributed, new ideas are needed to maintain the speed and accuracy of optimization. Operator splitting, which expresses the function to minimize as the sum of two convex functions, one of which is smooth and the other non-differentiable, is an idea that has produced to new first-order optimization methods. This project explores operator splitting with second-order optimization methods, which have faster convergence to the minimum. The focus is on large, distributed, and streaming data sets, so that the resulting general-purpose numerical solvers and embedded systems implementations can support optimization in cyberphysical systems and the Internet-of-Things. The project has as priority the active engagement and training of students and researchers, with specific emphasis on the inclusion of women and under-represented minority groups. This project not only involves collaboration across three top-tier American universities, but also with European research institute, KU Leuven. In specific, this research project seeks to interpret existing methods for structured convex optimization (such as the celebrated ADMM algorithm) as gradient methods applied to specific functions arising from the original problem formulation, and interpret of operator-splitting techniques as fixed point iterations for appropriately selected operators. A key theoretical foundation is the introduction of new envelope functions (smooth upper approximations possessing the same sets of solutions) that can be used as merit functions for variable-metric backtracking line-search. To conclude, a principal focus of the project is to design distributed asynchronous methods applicable to large-scale multi-agent cyberphysical systems that involve big data and impose stringent real-time constraints for decision-making. In this purview, the goal is to deliver methods that will outperform current state-of-the-art in terms of (a) speed of computations, (b) scalability with big data sizes, (c) robustness to various types of uncertainty, and, most topically, (d) distributed asynchronous implementation over networks in real-time. The merits will be illustrated in the context of applications in signal processing, control, machine learning and robotics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.