CIF: Small: Collaborative Research: Acceleration Algorithms for Large-scale Nonconvex Optimization

非凸优化问题在数据科学、机器学习和人工智能中无处不在。非凸性、模型参数的高维性和不确定数据的大量存在给这些问题的解决带来了巨大的挑战。虽然已经提出了一些流行的方法来加速求解实际大规模问题的优化算法，但这些算法对于非凸问题并不一定收敛，有些算法甚至在凸设置下也不收敛。本项目的主要目标是为设计具有可证明的理论收敛保证和卓越的实际性能的大规模非凸优化加速算法开发原则性方法。所开发的算法将适用于各个领域的大数据问题，包括深度学习、计算机视觉、医学图像处理、社交网络学习等。该项目将设计新颖、快速、可扩展的加速算法，用于解决各种大规模非凸问题，包括约束、复合和鞍点优化问题。这将包括受Nesterov方法启发的新型直接加速方法的发展，以及通过近点方法解决不同类型问题的间接加速方法。这些加速方法的性能将与随机化方法相结合，以提高其在数据维度和体积上的可扩展性。对大规模数据分析中出现的应用问题进行全面的数值验证。该项目将有助于优化与数据科学的综合，并将纳入课程开发，以及学生和未来大数据研究人员和实践者的培训。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Non-convex optimization problems are ubiquitous in data science, machine learning and artificial intelligence. Non-convexity, together with the high dimension of the model parameters and the large volume of uncertain data, presents significant challenges for solving these problems. Although popular methods have been proposed to speed up optimization algorithms for solving practical large-scale problems, these algorithms do not necessarily converge for non-convex problems, and some of them do not even converge in the convex setting. The primary goal of this project is to develop principled approaches for designing acceleration algorithms with provable theoretical convergence guarantees and superior practical performance for large-scale non-convex optimization. The developed algorithms will be applicable to big data problems in various domains, including deep learning, computer vision, medical image processing, social network learning, etc. This project will design novel, fast, and scalable acceleration algorithms for solving a variety of large-scale non-convex problems including constrained, composite, and saddle point optimization problems. This will include the development of both novel direct acceleration methods inspired by Nesterov's approach, and of indirect acceleration methods via proximal point methods for different types of problems. The performance of these acceleration methods will be explored when combined with randomization methods in order to enhance their scalability with data dimension and volume. Comprehensive numerical validations will be conducted for application problems arising in large-scale data analysis. This project will contribute to a synthesis of optimization with data science, and will be incorporated into curriculum development, and in the training of students and future big data researchers and practitioners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.