CIF: Small: Collaborative Research: Acceleration Algorithms for Large-scale Nonconvex Optimization

点击翻译按钮获取中文摘要

Non-convex optimization problems are ubiquitous in data science, machine learning and artificial intelligence. Non-convexity, together with the high dimension of the model parameters and the large volume of uncertain data, presents significant challenges for solving these problems. Although popular methods have been proposed to speed up optimization algorithms for solving practical large-scale problems, these algorithms do not necessarily converge for non-convex problems, and some of them do not even converge in the convex setting. The primary goal of this project is to develop principled approaches for designing acceleration algorithms with provable theoretical convergence guarantees and superior practical performance for large-scale non-convex optimization. The developed algorithms will be applicable to big data problems in various domains, including deep learning, computer vision, medical image processing, social network learning, etc.This project will design novel, fast, and scalable acceleration algorithms for solving a variety of large-scale non-convex problems including constrained, composite, and saddle point optimization problems. This will include the development of both novel direct acceleration methods inspired by Nesterov's approach, and of indirect acceleration methods via proximal point methods for different types of problems. The performance of these acceleration methods will be explored when combined with randomization methods in order to enhance their scalability with data dimension and volume. Comprehensive numerical validations will be conducted for application problems arising in large-scale data analysis. This project will contribute to a synthesis of optimization with data science, and will be incorporated into curriculum development, and in the training of students and future big data researchers and practitioners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.