AF: Small: Collaborative Research: Distributed Quasi-Newton Methods for Nonsmooth Optimization

优化，即找到产生最小输出的数学函数的输入，是云中智能设备或应用程序中许多进步背后的主力算法。随着数据变得越来越大，越来越分散，需要新的想法来保持优化的速度和准确性。算子分裂是一种产生新的一阶优化方法的思想，它将函数最小化表示为两个凸函数之和，其中一个凸函数是光滑的，另一个凸函数是不可微的。 这个项目探讨了二阶优化方法的算子分裂，它具有更快的收敛到最小值。 重点是大型，分布式和流式数据集，因此产生的通用数值求解器和嵌入式系统实现可以支持网络物理系统和物联网的优化。 该项目的优先事项是学生和研究人员的积极参与和培训，特别强调妇女和代表性不足的少数群体的参与。该项目不仅涉及美国三所顶级大学的合作，还涉及欧洲研究机构KU Leuven。具体而言，本研究项目旨在将现有的结构化凸优化方法（如著名的ADMM算法）解释为应用于原始问题公式中的特定函数的梯度方法，并将算子分裂技术解释为适当选择的算子的不动点迭代。 一个关键的理论基础是引入新的包络函数（光滑上近似拥有相同的解决方案集），可以用作可变度量回溯线搜索的价值函数。最后，该项目的主要重点是设计分布式异步方法适用于大规模的多代理网络物理系统，涉及大数据，并施加严格的实时约束的决策。在此范围内，目标是提供在以下方面优于当前最先进技术的方法：（a）计算速度，（B）大数据大小的可扩展性，（c）对各种类型的不确定性的鲁棒性，以及（d）实时网络上的分布式异步实现。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Optimization, which finds the inputs to a mathematical function that produce the minimum output, is a workhorse algorithm behind many of the advances in smart devices or applications in the cloud. As data gets larger and more distributed, new ideas are needed to maintain the speed and accuracy of optimization. Operator splitting, which expresses the function to minimize as the sum of two convex functions, one of which is smooth and the other non-differentiable, is an idea that has produced to new first-order optimization methods. This project explores operator splitting with second-order optimization methods, which have faster convergence to the minimum. The focus is on large, distributed, and streaming data sets, so that the resulting general-purpose numerical solvers and embedded systems implementations can support optimization in cyberphysical systems and the Internet-of-Things. The project has as priority the active engagement and training of students and researchers, with specific emphasis on the inclusion of women and under-represented minority groups. This project not only involves collaboration across three top-tier American universities, but also with European research institute, KU Leuven. In specific, this research project seeks to interpret existing methods for structured convex optimization (such as the celebrated ADMM algorithm) as gradient methods applied to specific functions arising from the original problem formulation, and interpret of operator-splitting techniques as fixed point iterations for appropriately selected operators. A key theoretical foundation is the introduction of new envelope functions (smooth upper approximations possessing the same sets of solutions) that can be used as merit functions for variable-metric backtracking line-search. To conclude, a principal focus of the project is to design distributed asynchronous methods applicable to large-scale multi-agent cyberphysical systems that involve big data and impose stringent real-time constraints for decision-making. In this purview, the goal is to deliver methods that will outperform current state-of-the-art in terms of (a) speed of computations, (b) scalability with big data sizes, (c) robustness to various types of uncertainty, and, most topically, (d) distributed asynchronous implementation over networks in real-time. The merits will be illustrated in the context of applications in signal processing, control, machine learning and robotics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.